Sep 10 15:59:44.432746 ip-10-0-143-169 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 15:59:44.432760 ip-10-0-143-169 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 15:59:44.432770 ip-10-0-143-169 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 15:59:44.433021 ip-10-0-143-169 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 15:59:55.751804 ip-10-0-143-169 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 15:59:55.751820 ip-10-0-143-169 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 246c2b4434a646c7b86a9f907c6eefb3 --
Sep 10 16:01:44.661629 ip-10-0-143-169 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 16:01:45.072437 ip-10-0-143-169 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:01:45.072437 ip-10-0-143-169 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 16:01:45.072437 ip-10-0-143-169 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:01:45.072437 ip-10-0-143-169 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 16:01:45.072437 ip-10-0-143-169 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:01:45.074787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.074707    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 16:01:45.082243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082221    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.082243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082238    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.082243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082243    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.082243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082247    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.082243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082250    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082254    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082257    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082260    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082263    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082266    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082269    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082272    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082275    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082278    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082281    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082284    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082287    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082290    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082292    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082295    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082298    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082301    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082303    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082306    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.082460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082308    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082311    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082315    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082319    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082322    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082338    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082341    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082344    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082347    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082350    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082353    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082355    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082358    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082360    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082363    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082366    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082369    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082372    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082375    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082378    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.082954 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082381    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082383    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082386    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082388    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082392    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082396    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082399    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082402    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082405    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082407    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082410    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082413    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082415    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082418    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082421    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082423    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082426    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082428    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082431    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.083544 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082433    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082436    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082438    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082441    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082443    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082446    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082448    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082452    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082454    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082457    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082460    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082463    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082466    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082469    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082472    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082475    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082478    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082480    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082483    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082485    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.084018 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082488    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082492    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082496    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082499    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082503    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082506    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082509    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082512    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082515    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082518    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082521    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082524    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082527    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082530    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082532    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.082535    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084033    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084041    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084045    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.084546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084048    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084051    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084053    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084056    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084058    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084061    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084063    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084066    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084069    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084071    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084074    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084076    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084079    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084082    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084085    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084088    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084091    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084094    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084097    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084099    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.085004 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084102    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084105    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084107    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084110    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084113    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084115    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084118    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084120    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084123    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084125    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084128    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084130    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084133    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084135    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084138    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084140    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084143    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084145    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084148    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084150    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.085511 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084153    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084157    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084161    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084165    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084168    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084171    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084173    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084178    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084181    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084184    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084187    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084190    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084193    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084196    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084198    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084201    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084203    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084206    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084211    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.085997 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084214    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084217    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084220    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084223    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084225    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084228    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084230    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084233    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084235    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084238    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084240    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084242    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084246    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084249    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084252    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084254    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084257    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084259    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084262    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.086488 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084266    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084270    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084274    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084277    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084279    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084282    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084285    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084287    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084290    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084292    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084295    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084297    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084300    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084302    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084305    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084307    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084310    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084312    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084696    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084702    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.086966 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084704    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084707    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084710    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084712    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084715    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084720    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084723    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084726    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084728    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084731    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084734    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084737    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084739    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084742    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084745    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084747    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084750    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084753    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084756    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084758    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.087481 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084761    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084763    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084766    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084770    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084773    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084776    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084781    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084784    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084787    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084789    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084792    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084794    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084797    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084799    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084802    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084804    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084807    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084812    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.087968 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084816    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084818    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084821    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084823    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084826    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084829    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084831    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084834    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084836    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084839    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084842    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084844    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084847    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084849    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084852    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084855    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084857    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084860    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084862    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.088426 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084867    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084870    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084873    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084876    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084879    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084882    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084884    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084887    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084889    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084891    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084894    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084897    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084901    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084904    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084906    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084909    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084911    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084913    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084916    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084919    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.088895 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084921    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084923    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084926    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084929    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084932    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084935    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084937    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084940    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084942    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084945    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084947    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084950    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084952    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084955    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084957    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084959    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084962    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084964    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084967    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.084969    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085041    2574 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 16:01:45.089430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085074    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085080    2574 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085085    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085090    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085093    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085097    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085101    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085104    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085107    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085110    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085113    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085116    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085119    2574 flags.go:64] FLAG: --cgroup-root=""
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085122    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085125    2574 flags.go:64] FLAG: --client-ca-file=""
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085128    2574 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085131    2574 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085135    2574 flags.go:64] FLAG: --cluster-domain=""
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085137    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085141    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085143    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085147    2574 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085150    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 16:01:45.089934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085153    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085156    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085159    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085162    2574 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085165    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085167    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085170    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085174    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085177    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085180    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085183    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085186    2574 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085190    2574 flags.go:64] FLAG: --enable-server="true"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085193    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085197    2574 flags.go:64] FLAG: --event-burst="100"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085201    2574 flags.go:64] FLAG: --event-qps="50"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085204    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085207    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085210    2574 flags.go:64] FLAG: --eviction-hard=""
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085214    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085217    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085219    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085222    2574 flags.go:64] FLAG: --eviction-soft=""
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085225    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085228    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 16:01:45.090512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085232    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085235    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085238    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085240    2574 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085243    2574 flags.go:64] FLAG: --feature-gates=""
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085247    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085250    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085253    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085256    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085259    2574 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085262    2574 flags.go:64] FLAG: --help="false"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085265    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085269    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085272    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085275    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085278    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085281    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085284    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085287    2574 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085290    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085292    2574 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085295    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085298    2574 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085301    2574 flags.go:64] FLAG: --kube-reserved=""
Sep 10 16:01:45.091146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085308    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085310    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085313    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085316    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085319    2574 flags.go:64] FLAG: --lock-file=""
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085322    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085339    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085342    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085348    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085351    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085354    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085357    2574 flags.go:64] FLAG: --logging-format="text"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085360    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085363    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085366    2574 flags.go:64] FLAG: --manifest-url=""
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085368    2574 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085373    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085376    2574 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085380    2574 flags.go:64] FLAG: --max-pods="110"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085383    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085386    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085389    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085392    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085396    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085399    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 16:01:45.091751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085402    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085409    2574 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085412    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085415    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085418    2574 flags.go:64] FLAG: --pod-cidr=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085421    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085426    2574 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085428    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085432    2574 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085437    2574 flags.go:64] FLAG: --port="10250"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085440    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085442    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05cc60a606ea49fd8"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085446    2574 flags.go:64] FLAG: --qos-reserved=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085448    2574 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085451    2574 flags.go:64] FLAG: --register-node="true"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085454    2574 flags.go:64] FLAG: --register-with-taints=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085458    2574 flags.go:64] FLAG: --registry-burst="10"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085462    2574 flags.go:64] FLAG: --registry-qps="5"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085465    2574 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085468    2574 flags.go:64] FLAG: --reserved-memory=""
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085472    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085475    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085477    2574 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085480    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085483    2574 flags.go:64] FLAG: --runonce="false"
Sep 10 16:01:45.092364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085486    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085489    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085492    2574 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085495    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085498    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085500    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085503    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085506    2574 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085509    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085512    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085517    2574 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085520    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085523    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085526    2574 flags.go:64] FLAG: --system-cgroups=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085528    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085534    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085538    2574 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085540    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085544    2574 flags.go:64] FLAG: --tls-min-version=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085547    2574 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085550    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085553    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085556    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085559    2574 flags.go:64] FLAG: --v="2"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085562    2574 flags.go:64] FLAG: --version="false"
Sep 10 16:01:45.092996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085567    2574 flags.go:64] FLAG: --vmodule=""
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085571    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.085574    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085687    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085690    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085693    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085696    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085699    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085702    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085705    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085708    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085711    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085714    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085716    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085719    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085722    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085725    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085729    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085731    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085734    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.093627 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085736    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085739    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085741    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085746    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085748    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085751    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085769    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085773    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085776    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085779    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085782    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085785    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085787    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085791    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085794    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085797    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085799    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085802    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085804    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.094108 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085807    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085811    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085815    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085818    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085821    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085823    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085826    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085828    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085831    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085833    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085837    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085840    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085843    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085845    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085847    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085850    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085854    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085857    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085859    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085862    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.094602 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085865    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085867    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085870    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085872    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085876    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085880    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085882    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085889    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085892    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085895    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085899    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085903    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085906    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085910    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085913    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085916    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085918    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085921    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085923    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085926    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.095095 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085928    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085931    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085934    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085937    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085940    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085942    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085945    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085947    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085951    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085954    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085956    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085959    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085961    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085964    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085966    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085969    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085971    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085975    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085979    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.095634 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085982    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085985    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085988    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.085991    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.086603    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.093594    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.093609    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.094426    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.094434    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094489    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094493    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094498    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094501    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094503    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.096137 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094506    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094509    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094511    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094514    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094516    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094519    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094522    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094524    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094527    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094530    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094533    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094536    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094540    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094543    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094546    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094549    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094553    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094557    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094560    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.096505 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094563    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094565    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094568    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094571    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094574    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094577    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094580    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094582    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094585    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094587    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094590    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094592    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094595    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094597    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094600    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094602    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094605    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094607    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094610    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094613    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.096963 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094615    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094618    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094620    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094623    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094625    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094628    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094630    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094633    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094635    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094638    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094640    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094643    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094645    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094648    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094650    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094653    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094656    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094659    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094662    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094664    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.097460 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094667    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094669    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094673    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094676    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094679    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094682    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094684    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094687    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094697    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094700    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094702    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094704    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094707    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094709    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094712    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094715    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094717    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094720    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094722    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.097949 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094725    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094727    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094730    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094732    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094734    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094737    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094740    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094742    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094745    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094748    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094750    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094753    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094756    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094759    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094762    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.098438 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094764    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.094769    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094876    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094880    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094883    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094886    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094888    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094891    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094893    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094896    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094898    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094901    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094903    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094906    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094908    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:01:45.098813 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094911    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094913    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094916    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094918    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094921    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094923    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094926    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094929    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094931    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094933    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094936    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094939    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094941    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094944    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094946    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094950    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094952    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094955    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094957    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094959    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:01:45.099174 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094962    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094964    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094966    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094969    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094971    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094974    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094976    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094979    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094981    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094983    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094986    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094988    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094990    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094993    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094995    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.094998    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095001    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095003    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095006    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095008    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:01:45.099684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095011    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095014    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095016    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095019    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095021    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095024    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095026    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095028    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095031    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095034    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095036    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095040    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095043    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095046    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095049    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095052    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095054    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095057    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095060    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095062    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:01:45.100168 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095064    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095067    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095070    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095074    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095077    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095079    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095082    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095085    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095087    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095090    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095092    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095095    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095099    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095102    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095105    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095107    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095110    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095113    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095115    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:01:45.100657 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095118    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095120    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095123    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095125    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095128    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095130    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:45.095132    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.095137    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.095231    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.098189    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.099142    2574 server.go:1019] "Starting client certificate rotation"
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.099241    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 16:01:45.101143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.099274    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 16:01:45.102633 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.102620    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 16:01:45.104676 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.104657    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 16:01:45.115347 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.115320    2574 log.go:25] "Validated CRI v1 runtime API"
Sep 10 16:01:45.118727 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.118703    2574 log.go:25] "Validated CRI v1 image API"
Sep 10 16:01:45.119508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.119494    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 16:01:45.121925 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.121908    2574 fs.go:135] Filesystem UUIDs: map[3ad98717-ec32-4ec6-be2d-346aba847980:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9eed3b3a-f0f2-4038-a458-610071861f33:/dev/nvme0n1p3]
Sep 10 16:01:45.121984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.121926    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 16:01:45.127846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.127729    2574 manager.go:216] Machine: {Timestamp:2026-09-10 16:01:45.126362868 +0000 UTC m=+0.359310961 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098804 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d20d4c61d850b562d2b13fbaa92b1 SystemUUID:ec2d20d4-c61d-850b-562d-2b13fbaa92b1 BootID:246c2b44-34a6-46c7-b86a-9f907c6eefb3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:2e:c8:37:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:2e:c8:37:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:e3:b3:a5:26:51 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 16:01:45.127846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.127836    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 16:01:45.127984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.127910    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 16:01:45.129368 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.129355    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 16:01:45.130526 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.130503    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 16:01:45.130656 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.130531    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-169.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 16:01:45.131423 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.131413    2574 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 16:01:45.131460 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.131425    2574 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 16:01:45.131460 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.131440    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 16:01:45.132289 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.132280    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 16:01:45.134121 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.134112    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 16:01:45.138305 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138295    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 16:01:45.138432 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138423    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 16:01:45.138467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138451    2574 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 16:01:45.138495 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138467    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 16:01:45.138495 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138481    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 16:01:45.138495 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138495    2574 kubelet.go:398] "Adding apiserver pod source"
Sep 10 16:01:45.138578 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138506    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 16:01:45.138877 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138866    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 16:01:45.138927 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.138884    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 16:01:45.141457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.141441    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 16:01:45.144174 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144162    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 16:01:45.144226 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144191    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 16:01:45.144363 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144353    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 16:01:45.144394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144367    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 16:01:45.144394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144374    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 16:01:45.144394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144379    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 16:01:45.144394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144387    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 16:01:45.144394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.144392    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 16:01:45.145074 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145065    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 16:01:45.145103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145074    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 16:01:45.145103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145081    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 16:01:45.145103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145086    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 16:01:45.145103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145099    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 16:01:45.145203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145106    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 16:01:45.145980 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145970    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 16:01:45.146017 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.145980    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 16:01:45.148716 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.148702    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-169.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 16:01:45.149298 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.149280    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 16:01:45.149343 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.149277    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 16:01:45.149681 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.149669    2574 server.go:1268] "Started kubelet"
Sep 10 16:01:45.149762 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.149738    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 16:01:45.149930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.149867    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 16:01:45.154039 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.154012    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 16:01:45.154784 ip-10-0-143-169 systemd[1]: Started Kubernetes Kubelet.
Sep 10 16:01:45.156502 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.156356    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 16:01:45.156554 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.156533    2574 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 16:01:45.157303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.157286    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 16:01:45.161983 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.160905    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-169.ec2.internal.18d4010c7eb02a44  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-169.ec2.internal,UID:ip-10-0-143-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-169.ec2.internal,},FirstTimestamp:2026-09-10 16:01:45.149639236 +0000 UTC m=+0.382587329,LastTimestamp:2026-09-10 16:01:45.149639236 +0000 UTC m=+0.382587329,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-169.ec2.internal,}"
Sep 10 16:01:45.162935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.162918    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 16:01:45.163562 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.163542    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 16:01:45.163768 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.163749    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.164414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164400    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 16:01:45.164414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164406    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 16:01:45.164572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164416    2574 factory.go:55] Registering systemd factory
Sep 10 16:01:45.164572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164424    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 16:01:45.164572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164431    2574 factory.go:223] Registration of the systemd container factory successfully
Sep 10 16:01:45.164572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164512    2574 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 16:01:45.164572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164519    2574 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164637    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kzwjm"
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164672    2574 factory.go:153] Registering CRI-O factory
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164685    2574 factory.go:223] Registration of the crio container factory successfully
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164729    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164746    2574 factory.go:103] Registering Raw factory
Sep 10 16:01:45.164787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.164763    2574 manager.go:1196] Started watching for new ooms in manager
Sep 10 16:01:45.165811 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.165797    2574 manager.go:349] Starting recovery of all containers
Sep 10 16:01:45.172389 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.172360    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 16:01:45.172544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.172532    2574 manager.go:354] Recovery completed
Sep 10 16:01:45.173032 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.173007    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 16:01:45.174874 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.174854    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kzwjm"
Sep 10 16:01:45.178025 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.178012    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.179640 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.179623    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.179719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.179653    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.179719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.179669    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.180336 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.180208    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 16:01:45.180391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.180341    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 16:01:45.180391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.180357    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 16:01:45.180456 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.180210    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.181051 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.181038    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.181107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.181062    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.181107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.181072    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.181500 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.181419    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-169.ec2.internal.18d4010c8079eee1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-169.ec2.internal,UID:ip-10-0-143-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-169.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-169.ec2.internal,},FirstTimestamp:2026-09-10 16:01:45.179639521 +0000 UTC m=+0.412587615,LastTimestamp:2026-09-10 16:01:45.179639521 +0000 UTC m=+0.412587615,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-169.ec2.internal,}"
Sep 10 16:01:45.182163 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.182147    2574 policy_none.go:49] "None policy: Start"
Sep 10 16:01:45.182219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.182170    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 16:01:45.182219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.182183    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 16:01:45.186607 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.186591    2574 policy_none.go:47] "Start"
Sep 10 16:01:45.209892 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.209878    2574 manager.go:337] "Starting Device Plugin manager"
Sep 10 16:01:45.209969 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.209913    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 16:01:45.209969 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.209922    2574 server.go:87] "Starting device plugin registration server"
Sep 10 16:01:45.210218 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.210205    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 16:01:45.210280 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.210220    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 16:01:45.210480 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.210338    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 16:01:45.210480 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.210422    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 16:01:45.210480 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.210431    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 16:01:45.210983 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.210963    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 16:01:45.211066 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.211022    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.258497 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.258470    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 16:01:45.260424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.259673    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 16:01:45.260424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.259711    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 16:01:45.260424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.259729    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 16:01:45.260424 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.259810    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 16:01:45.262026 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.262007    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:01:45.311170 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.311151    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.312101 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.312085    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.312191 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.312112    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.312191 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.312125    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.312191 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.312153    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.320565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.320550    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.320639 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.320570    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-169.ec2.internal\": node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.330919 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.330848    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.360908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.360885    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"]
Sep 10 16:01:45.360976 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.360957    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.361780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.361764    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.361846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.361790    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.361846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.361799    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.363848 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.363836    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.363979 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.363965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.364034 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364001    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.364574 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364557    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.364664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364587    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.364664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364604    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.364664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364558    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.364811 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364672    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.364811 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.364687    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.365368 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.365352    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e52fe6a08309506396b2cf1a56b350d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-169.ec2.internal\" (UID: \"4e52fe6a08309506396b2cf1a56b350d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.365414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.365375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.365414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.365391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.366586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.366571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.366665 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.366599    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:01:45.367297 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.367275    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:01:45.367388 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.367301    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:01:45.367388 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.367315    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:01:45.392050 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.392028    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-169.ec2.internal\" not found" node="ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.396294 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.396277    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-169.ec2.internal\" not found" node="ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.431145 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.431128    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.465565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.465624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e52fe6a08309506396b2cf1a56b350d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-169.ec2.internal\" (UID: \"4e52fe6a08309506396b2cf1a56b350d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.465624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.465680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465624    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.465680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6aefb6f6c14fbe159c01d407c6986b03-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal\" (UID: \"6aefb6f6c14fbe159c01d407c6986b03\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.465680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.465669    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4e52fe6a08309506396b2cf1a56b350d-config\") pod \"kube-apiserver-proxy-ip-10-0-143-169.ec2.internal\" (UID: \"4e52fe6a08309506396b2cf1a56b350d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.531772 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.531755    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.632390 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.632307    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.694906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.694874    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.699539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:45.699510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:45.733430 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.733402    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.833875 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.833844    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:45.934418 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:45.934361    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:46.036841 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:46.036811    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:46.101098 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.101060    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 16:01:46.101518 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.101222    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 16:01:46.139225 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:46.139198    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-169.ec2.internal\" not found"
Sep 10 16:01:46.179105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.179072    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 15:56:45 +0000 UTC" deadline="2028-04-17 09:59:05.585678554 +0000 UTC"
Sep 10 16:01:46.179105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.179099    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14033h57m19.406582487s"
Sep 10 16:01:46.223607 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.223547    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:01:46.228338 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.228321    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:01:46.264127 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.264109    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:46.276098 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.276077    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 16:01:46.277027 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.277014    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal"
Sep 10 16:01:46.288770 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.288748    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 16:01:46.444382 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:46.444348    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e52fe6a08309506396b2cf1a56b350d.slice/crio-1f720f67777a95f817337204400da242fb2369853a9c063ed64b439651d07fac WatchSource:0}: Error finding container 1f720f67777a95f817337204400da242fb2369853a9c063ed64b439651d07fac: Status 404 returned error can't find the container with id 1f720f67777a95f817337204400da242fb2369853a9c063ed64b439651d07fac
Sep 10 16:01:46.448894 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.448874    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:01:46.594458 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:46.594429    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aefb6f6c14fbe159c01d407c6986b03.slice/crio-bd2deb9ee1fc145b32b2a677090cdc34367ea29cea717f949246c505b7b147f7 WatchSource:0}: Error finding container bd2deb9ee1fc145b32b2a677090cdc34367ea29cea717f949246c505b7b147f7: Status 404 returned error can't find the container with id bd2deb9ee1fc145b32b2a677090cdc34367ea29cea717f949246c505b7b147f7
Sep 10 16:01:46.677276 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:46.677248    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:01:47.141553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.141519    2574 apiserver.go:52] "Watching apiserver"
Sep 10 16:01:47.153094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.153067    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 16:01:47.154224 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.154200    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal","openshift-multus/multus-additional-cni-plugins-sx9sp","openshift-multus/multus-694zm","openshift-multus/network-metrics-daemon-lns56","openshift-network-diagnostics/network-check-target-c67tz","openshift-network-operator/iptables-alerter-8qpkg","openshift-ovn-kubernetes/ovnkube-node-pskgr","kube-system/global-pull-secret-syncer-c7krj","kube-system/konnectivity-agent-vp6vj","kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7","openshift-cluster-node-tuning-operator/tuned-jz2lc","openshift-image-registry/node-ca-ms7g8"]
Sep 10 16:01:47.157082 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.157063    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.159433 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.159413    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.160145 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.160125    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 16:01:47.160225 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.160157    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wfqlt\""
Sep 10 16:01:47.160225 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.160133    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 16:01:47.161748 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.161731    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.161989 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.161966    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 16:01:47.161989 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.161981    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 16:01:47.162256 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.162234    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 16:01:47.162439 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.162411    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.162513 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.162468    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmw4p\""
Sep 10 16:01:47.162577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.162518    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.163956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.163929    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 16:01:47.164556 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.164050    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.164623 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.164608    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 16:01:47.164893 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.164876    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6hhzh\""
Sep 10 16:01:47.166518 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.166499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:47.166617 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.166582    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:47.166678 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.166651    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 16:01:47.167123 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.167104    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.167213 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.167131    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nt2cf\""
Sep 10 16:01:47.167277 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.167265    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.168838 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.168815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.171238 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.171221    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.171850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.171828    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.171955 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.171853    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.171955 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.171873    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z9t9p\""
Sep 10 16:01:47.171955 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.171887    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 16:01:47.173608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.173594    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.173959 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.173937    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.174036 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.173994    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lg7rc\""
Sep 10 16:01:47.174036 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.174019    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.176108 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.176508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-system-cni-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.176605 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.176605 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176605 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-cnibin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176625    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-cni-binary-copy\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-socket-dir-parent\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176675    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-bin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-kubelet\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176742    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-multus-daemon-config\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.176772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-system-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176799    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-conf-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-agent-certs\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.177064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-hostroot\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176887    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.177064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.176944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-netns\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-etc-kubernetes\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177198    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 16:01:47.177303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrnnw\" (UniqueName: \"kubernetes.io/projected/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-kube-api-access-jrnnw\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.177303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-multus-certs\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-konnectivity-ca\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-kube-api-access-x49nn\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177494    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-multus\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177539    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j647n\" (UniqueName: \"kubernetes.io/projected/96176111-14ab-4cfb-96cc-7884315c2410-kube-api-access-j647n\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-k8s-cni-cncf-io\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177660    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-host\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.177693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177664    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177695    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-serviceca\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cnibin\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177826    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-688sw\""
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.178040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.177958    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.178737 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.178714    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.178823 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.178792    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:47.181113 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.181091    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 16:01:47.181638 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.181622    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 16:01:47.182011 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.181990    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 16:01:47.182174 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.182150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.182313 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.182294    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:47.182856 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.182833    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 16:01:47.182960 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.182858    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dk7dz\""
Sep 10 16:01:47.182960 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.182917    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 16:01:47.183509 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.183484    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 16:01:47.185018 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.185002    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 16:01:47.203354 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.203336    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6gp56"
Sep 10 16:01:47.214986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.214969    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6gp56"
Sep 10 16:01:47.264467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.264426    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal" event={"ID":"6aefb6f6c14fbe159c01d407c6986b03","Type":"ContainerStarted","Data":"bd2deb9ee1fc145b32b2a677090cdc34367ea29cea717f949246c505b7b147f7"}
Sep 10 16:01:47.265082 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.265062    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 16:01:47.265565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.265541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal" event={"ID":"4e52fe6a08309506396b2cf1a56b350d","Type":"ContainerStarted","Data":"1f720f67777a95f817337204400da242fb2369853a9c063ed64b439651d07fac"}
Sep 10 16:01:47.278234 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-hostroot\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278320 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-dbus\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.278320 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.278320 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-node-log\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.278447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278356    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-hostroot\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-etc-kubernetes\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrnnw\" (UniqueName: \"kubernetes.io/projected/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-kube-api-access-jrnnw\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.278447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-etc-kubernetes\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mpc4\" (UniqueName: \"kubernetes.io/projected/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-kube-api-access-5mpc4\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.278647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-conf\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.278647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278489    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.278647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278513    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.278647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278559    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-env-overrides\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.278647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-multus-certs\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-multus\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278698    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-multus\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278670    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-kubelet\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278733    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-multus-certs\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278772    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-systemd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j647n\" (UniqueName: \"kubernetes.io/projected/96176111-14ab-4cfb-96cc-7884315c2410-kube-api-access-j647n\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-k8s-cni-cncf-io\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.278865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-k8s-cni-cncf-io\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.278980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-host\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-device-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279040    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a782a69a-5e9e-4eec-9add-50617b03773d-ovn-node-metrics-cert\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-script-lib\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-serviceca\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-modprobe-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-socket-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-log-socket\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-ovn\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.279219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279232    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-cni-binary-copy\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-socket-dir-parent\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-bin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279399    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-socket-dir-parent\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-kubelet\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-cni-bin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-conf-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279479    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-var-lib-kubelet\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysconfig\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-conf-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-lib-modules\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.279600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-tuned\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k5w7\" (UniqueName: \"kubernetes.io/projected/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-kube-api-access-7k5w7\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-bin\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279671    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-agent-certs\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-netns\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279719    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-cni-binary-copy\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-iptables-alerter-script\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279786    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-sys\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279823    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-konnectivity-ca\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-kube-api-access-x49nn\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-sys-fs\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279894    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6ks4\" (UniqueName: \"kubernetes.io/projected/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kube-api-access-r6ks4\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279818    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-host-run-netns\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.280105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.279918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-slash\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280104    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-netns\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280142    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280290    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-serviceca\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280348    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-host\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280440    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-konnectivity-ca\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-host\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280461    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-run\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280495    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-kubelet-config\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-var-lib-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-etc-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-config\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hxqb\" (UniqueName: \"kubernetes.io/projected/a782a69a-5e9e-4eec-9add-50617b03773d-kube-api-access-6hxqb\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280660    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-host-slash\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-tmp\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.280833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-registration-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-netd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cnibin\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cnibin\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-kubernetes\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280915    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-var-lib-kubelet\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg95b\" (UniqueName: \"kubernetes.io/projected/d1e89918-a035-4d0c-9100-2db3b17e4310-kube-api-access-mg95b\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-system-cni-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.280981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281077    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-system-cni-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281125    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281136    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-cnibin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281149    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-multus-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.281566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-systemd\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281202    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-cnibin\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-multus-daemon-config\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-system-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-systemd-units\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281380    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96176111-14ab-4cfb-96cc-7884315c2410-system-cni-dir\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.282102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.281689    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96176111-14ab-4cfb-96cc-7884315c2410-multus-daemon-config\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.283765 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.283748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8ebdd40-30cf-4864-9b3a-1827cbc9d72a-agent-certs\") pod \"konnectivity-agent-vp6vj\" (UID: \"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a\") " pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.284915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.284884    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:01:47.290197 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.290171    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:01:47.290197 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.290192    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:01:47.290317 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.290206    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:47.290376 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.290325    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:01:47.790302541 +0000 UTC m=+3.023250637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:47.292543 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.292524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j647n\" (UniqueName: \"kubernetes.io/projected/96176111-14ab-4cfb-96cc-7884315c2410-kube-api-access-j647n\") pod \"multus-694zm\" (UID: \"96176111-14ab-4cfb-96cc-7884315c2410\") " pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.292634 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.292618    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrnnw\" (UniqueName: \"kubernetes.io/projected/9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c-kube-api-access-jrnnw\") pod \"node-ca-ms7g8\" (UID: \"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c\") " pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.293258 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.293241    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/ae9df9b4-3da2-464d-a95b-50b09e43ba8d-kube-api-access-x49nn\") pod \"multus-additional-cni-plugins-sx9sp\" (UID: \"ae9df9b4-3da2-464d-a95b-50b09e43ba8d\") " pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.381703 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-iptables-alerter-script\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.381703 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-sys\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381718    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-sys-fs\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6ks4\" (UniqueName: \"kubernetes.io/projected/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kube-api-access-r6ks4\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-slash\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-netns\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-sys\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381840    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-netns\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-sys-fs\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-slash\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381878    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.381906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-run\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-kubelet-config\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381926    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-run\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381938    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-var-lib-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-etc-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-config\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hxqb\" (UniqueName: \"kubernetes.io/projected/a782a69a-5e9e-4eec-9add-50617b03773d-kube-api-access-6hxqb\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.381993    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-kubelet-config\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-host-slash\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382035    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-var-lib-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382051    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-host-slash\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-etc-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-tmp\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-registration-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-registration-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.382214    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382223    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-netd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.382494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382264    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-kubernetes\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.382294    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:47.882268857 +0000 UTC m=+3.115216941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382317    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-kubernetes\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-var-lib-kubelet\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382380    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-netd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg95b\" (UniqueName: \"kubernetes.io/projected/d1e89918-a035-4d0c-9100-2db3b17e4310-kube-api-access-mg95b\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-systemd\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-var-lib-kubelet\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-systemd-units\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382495    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-systemd\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-dbus\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-systemd-units\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382562    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-node-log\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.383266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-dbus\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-config\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382626    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382601    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mpc4\" (UniqueName: \"kubernetes.io/projected/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-kube-api-access-5mpc4\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382659    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-node-log\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-conf\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-env-overrides\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-kubelet\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-systemd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382825    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-run-ovn-kubernetes\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.382833    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382850    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-kubelet\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-host\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382816    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysctl-conf\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382867    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-systemd\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.384178 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.382895    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:47.882878465 +0000 UTC m=+3.115826558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-host\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-device-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382951    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-device-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a782a69a-5e9e-4eec-9add-50617b03773d-ovn-node-metrics-cert\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.382996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-script-lib\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-modprobe-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-socket-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-log-socket\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383106    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-env-overrides\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383123    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-ovn\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383128    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-modprobe-d\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383162    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-openvswitch\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-run-ovn\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383174    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-log-socket\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysconfig\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-lib-modules\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383262    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-socket-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-tuned\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383307    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k5w7\" (UniqueName: \"kubernetes.io/projected/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-kube-api-access-7k5w7\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383317    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-sysconfig\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-bin\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383370    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-iptables-alerter-script\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383304    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383449    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a782a69a-5e9e-4eec-9add-50617b03773d-host-cni-bin\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-lib-modules\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.383578    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a782a69a-5e9e-4eec-9add-50617b03773d-ovnkube-script-lib\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.384581    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-tmp\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.385257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-etc-tuned\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.385773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.385624    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a782a69a-5e9e-4eec-9add-50617b03773d-ovn-node-metrics-cert\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.394369 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.394311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg95b\" (UniqueName: \"kubernetes.io/projected/d1e89918-a035-4d0c-9100-2db3b17e4310-kube-api-access-mg95b\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.394505 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.394408    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6ks4\" (UniqueName: \"kubernetes.io/projected/b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa-kube-api-access-r6ks4\") pod \"aws-ebs-csi-driver-node-n7gv7\" (UID: \"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.394505 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.394455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mpc4\" (UniqueName: \"kubernetes.io/projected/4b9c8e0f-377a-47c3-a3e9-d048a4a703ed-kube-api-access-5mpc4\") pod \"iptables-alerter-8qpkg\" (UID: \"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed\") " pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.395045 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.395024    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hxqb\" (UniqueName: \"kubernetes.io/projected/a782a69a-5e9e-4eec-9add-50617b03773d-kube-api-access-6hxqb\") pod \"ovnkube-node-pskgr\" (UID: \"a782a69a-5e9e-4eec-9add-50617b03773d\") " pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.395145 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.395129    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k5w7\" (UniqueName: \"kubernetes.io/projected/a5d651a9-97ae-4b8b-99d3-a2d8992e4612-kube-api-access-7k5w7\") pod \"tuned-jz2lc\" (UID: \"a5d651a9-97ae-4b8b-99d3-a2d8992e4612\") " pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.468656 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.468621    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:01:47.474237 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.474216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sx9sp"
Sep 10 16:01:47.479246 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.479225    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ebdd40_30cf_4864_9b3a_1827cbc9d72a.slice/crio-42ca624747caa27bfe066cb440c98208a9055258cc640e618bdb15ef393c314d WatchSource:0}: Error finding container 42ca624747caa27bfe066cb440c98208a9055258cc640e618bdb15ef393c314d: Status 404 returned error can't find the container with id 42ca624747caa27bfe066cb440c98208a9055258cc640e618bdb15ef393c314d
Sep 10 16:01:47.483445 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.483417    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9df9b4_3da2_464d_a95b_50b09e43ba8d.slice/crio-e17902cce05ea7b7ccaec710d33d8ca622ecd38f6a692b339ee9a4421d35ea6c WatchSource:0}: Error finding container e17902cce05ea7b7ccaec710d33d8ca622ecd38f6a692b339ee9a4421d35ea6c: Status 404 returned error can't find the container with id e17902cce05ea7b7ccaec710d33d8ca622ecd38f6a692b339ee9a4421d35ea6c
Sep 10 16:01:47.483668 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.483653    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-694zm"
Sep 10 16:01:47.488600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.488583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ms7g8"
Sep 10 16:01:47.493011 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.492989    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96176111_14ab_4cfb_96cc_7884315c2410.slice/crio-9f69e4d4c2a1ec1fc49222db68de529bf554d08b1cf76435e4c5924ae8fec3f0 WatchSource:0}: Error finding container 9f69e4d4c2a1ec1fc49222db68de529bf554d08b1cf76435e4c5924ae8fec3f0: Status 404 returned error can't find the container with id 9f69e4d4c2a1ec1fc49222db68de529bf554d08b1cf76435e4c5924ae8fec3f0
Sep 10 16:01:47.494362 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.494344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8qpkg"
Sep 10 16:01:47.498291 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.498254    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4fbf6c_bef9_40d1_ba57_83a6f1ecd27c.slice/crio-de6d589b7537070d5c7a6b0efb712e9b7ae27045900c4369ccf0706b474af979 WatchSource:0}: Error finding container de6d589b7537070d5c7a6b0efb712e9b7ae27045900c4369ccf0706b474af979: Status 404 returned error can't find the container with id de6d589b7537070d5c7a6b0efb712e9b7ae27045900c4369ccf0706b474af979
Sep 10 16:01:47.500444 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.500424    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jz2lc"
Sep 10 16:01:47.503542 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.503523    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9c8e0f_377a_47c3_a3e9_d048a4a703ed.slice/crio-f3ec67b28b4d5f81589e44531a8db3f37016f086b134cdb95f2dc6100910de2f WatchSource:0}: Error finding container f3ec67b28b4d5f81589e44531a8db3f37016f086b134cdb95f2dc6100910de2f: Status 404 returned error can't find the container with id f3ec67b28b4d5f81589e44531a8db3f37016f086b134cdb95f2dc6100910de2f
Sep 10 16:01:47.506604 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.506535    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7"
Sep 10 16:01:47.508649 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.508627    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d651a9_97ae_4b8b_99d3_a2d8992e4612.slice/crio-4b618cd716cee7e949a62bfaf2223bd7e9997173fe09dc8dddbd347505635939 WatchSource:0}: Error finding container 4b618cd716cee7e949a62bfaf2223bd7e9997173fe09dc8dddbd347505635939: Status 404 returned error can't find the container with id 4b618cd716cee7e949a62bfaf2223bd7e9997173fe09dc8dddbd347505635939
Sep 10 16:01:47.512740 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.512588    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:01:47.518985 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.518964    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a85a5d_4b7e_43f5_a302_d6392e3ac5aa.slice/crio-88d222614363d027a321404ed8e04a010075735a34282c91f4e7e45939e3804b WatchSource:0}: Error finding container 88d222614363d027a321404ed8e04a010075735a34282c91f4e7e45939e3804b: Status 404 returned error can't find the container with id 88d222614363d027a321404ed8e04a010075735a34282c91f4e7e45939e3804b
Sep 10 16:01:47.521811 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:01:47.521790    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda782a69a_5e9e_4eec_9add_50617b03773d.slice/crio-d45dbc83d1b97745c67131df1137c8e083658e6821ad0c2436bbfe5adfc03219 WatchSource:0}: Error finding container d45dbc83d1b97745c67131df1137c8e083658e6821ad0c2436bbfe5adfc03219: Status 404 returned error can't find the container with id d45dbc83d1b97745c67131df1137c8e083658e6821ad0c2436bbfe5adfc03219
Sep 10 16:01:47.886429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.886209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:47.886429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.886260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:47.886429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:47.886315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:47.886429 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886370    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886445    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:48.886423663 +0000 UTC m=+4.119371758 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886450    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886503    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:48.886486099 +0000 UTC m=+4.119434185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886587    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886615    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886628    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:47.886754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:47.886665    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:01:48.886653181 +0000 UTC m=+4.119601272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:48.219068 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.218902    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:56:47 +0000 UTC" deadline="2028-04-11 05:17:07.590566624 +0000 UTC"
Sep 10 16:01:48.219068 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.218940    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13885h15m19.37163151s"
Sep 10 16:01:48.261461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.260839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:48.261461 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.260992    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:48.271624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.271567    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jz2lc" event={"ID":"a5d651a9-97ae-4b8b-99d3-a2d8992e4612","Type":"ContainerStarted","Data":"4b618cd716cee7e949a62bfaf2223bd7e9997173fe09dc8dddbd347505635939"}
Sep 10 16:01:48.275781 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.275749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qpkg" event={"ID":"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed","Type":"ContainerStarted","Data":"f3ec67b28b4d5f81589e44531a8db3f37016f086b134cdb95f2dc6100910de2f"}
Sep 10 16:01:48.278678 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.278648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-694zm" event={"ID":"96176111-14ab-4cfb-96cc-7884315c2410","Type":"ContainerStarted","Data":"9f69e4d4c2a1ec1fc49222db68de529bf554d08b1cf76435e4c5924ae8fec3f0"}
Sep 10 16:01:48.281087 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.281061    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7" event={"ID":"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa","Type":"ContainerStarted","Data":"88d222614363d027a321404ed8e04a010075735a34282c91f4e7e45939e3804b"}
Sep 10 16:01:48.287913 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.287658    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ms7g8" event={"ID":"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c","Type":"ContainerStarted","Data":"de6d589b7537070d5c7a6b0efb712e9b7ae27045900c4369ccf0706b474af979"}
Sep 10 16:01:48.294390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.294364    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerStarted","Data":"e17902cce05ea7b7ccaec710d33d8ca622ecd38f6a692b339ee9a4421d35ea6c"}
Sep 10 16:01:48.302038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.301214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp6vj" event={"ID":"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a","Type":"ContainerStarted","Data":"42ca624747caa27bfe066cb440c98208a9055258cc640e618bdb15ef393c314d"}
Sep 10 16:01:48.304862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.304751    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"d45dbc83d1b97745c67131df1137c8e083658e6821ad0c2436bbfe5adfc03219"}
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.895089    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.895155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:48.895198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895366    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895407    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895422    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895508    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:01:50.895464979 +0000 UTC m=+6.128413062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895940    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.895991    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:50.895976072 +0000 UTC m=+6.128924160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.896055    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:48.896138 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:48.896090    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:50.896078903 +0000 UTC m=+6.129026991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:49.220068 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:49.219957    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 15:56:47 +0000 UTC" deadline="2028-04-03 11:18:36.735581718 +0000 UTC"
Sep 10 16:01:49.220068 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:49.220000    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13699h16m47.515586989s"
Sep 10 16:01:49.265664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:49.265624    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:49.265858 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:49.265831    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:49.265943 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:49.265928    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:49.266076 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:49.266057    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:50.260564 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:50.260528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:50.261059 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.260685    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:50.913263 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:50.913209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:50.913477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:50.913272    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:50.913477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:50.913306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:50.913477 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913461    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913481    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913551    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:54.913530592 +0000 UTC m=+10.146478690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913574    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:01:54.91356293 +0000 UTC m=+10.146511013 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913622    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913642    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:01:50.913658 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913655    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:50.913956 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:50.913702    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:01:54.913682928 +0000 UTC m=+10.146631029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:51.260414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:51.260322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:51.260594 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:51.260322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:51.260594 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:51.260507    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:51.261030 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:51.260946    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:52.260162 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:52.260129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:52.260355 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:52.260260    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:53.260612 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:53.260579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:53.261113 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:53.260582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:53.261113 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:53.260713    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:53.261113 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:53.260828    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:54.260638 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:54.260561    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:54.261120 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.260701    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:54.951398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:54.951462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:54.951496    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951562    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951618    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:54.951641 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951636    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:02.951616591 +0000 UTC m=+18.184564688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:01:54.952100 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951659    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:02.951647492 +0000 UTC m=+18.184595583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:01:54.952100 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951618    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:01:54.952100 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951681    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:01:54.952100 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951693    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:54.952100 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:54.951722    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:02:02.951713156 +0000 UTC m=+18.184661236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:01:55.261425 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:55.261344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:55.261942 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:55.261472    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:55.261942 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:55.261839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:55.261942 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:55.261926    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:56.260518 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:56.260478    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:56.260692 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:56.260618    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:57.260209 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:57.260170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:57.260209 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:57.260207    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:57.260671 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:57.260313    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:57.260671 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:57.260463    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:01:58.260541 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:58.260456    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:01:58.260925 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:58.260571    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:01:59.262994 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:59.262963    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:01:59.263408 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:59.263074    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:01:59.263408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:01:59.263143    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:01:59.263408 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:01:59.263255    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:00.260203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:00.260170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:00.260386 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:00.260296    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:01.260717 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:01.260685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:01.260717 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:01.260703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:01.261109 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:01.260783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:01.261109 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:01.260924    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:02.260134 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:02.260097    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:02.260415 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:02.260238    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:03.009483 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:03.009440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:03.009505    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:03.009538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009599    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009642    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009672    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009679    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:19.009658314 +0000 UTC m=+34.242606410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009690    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009696    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:19.009689267 +0000 UTC m=+34.242637347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009703    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:02:03.009952 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.009752    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:02:19.00973646 +0000 UTC m=+34.242684553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:02:03.260964 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:03.260886    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:03.261106 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.261007    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:03.261106 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:03.261055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:03.261214 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:03.261159    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:04.260396 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:04.260360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:04.260859 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:04.260487    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:05.261162 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:05.261126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:05.261557 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:05.261240    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:05.261557 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:05.261306    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:05.261557 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:05.261412    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:06.260025 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:06.259999    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:06.260185 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:06.260164    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:06.347391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:06.347197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal" event={"ID":"4e52fe6a08309506396b2cf1a56b350d","Type":"ContainerStarted","Data":"ba3a2ee5f6bd7d34bd471f9531643e48964201e73b52d5b556bf520dbcc21ab5"}
Sep 10 16:02:06.352447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:06.352358    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"2c8fcd59958d694ec875d23da6685ae0045288717fc1189bd0ae2eab991839f3"}
Sep 10 16:02:06.355172 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:06.355131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jz2lc" event={"ID":"a5d651a9-97ae-4b8b-99d3-a2d8992e4612","Type":"ContainerStarted","Data":"69f22bf122ba7387915e2f02846cc19ab86b8764313e493e49f962011c47e7fb"}
Sep 10 16:02:06.375290 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:06.375166    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jz2lc" podStartSLOduration=2.725767964 podStartE2EDuration="21.375145976s" totalImagesPullingTime="18.649378012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:06.374459521 +0000 UTC m=+21.607407624" watchObservedRunningTime="2026-09-10 16:02:06.375145976 +0000 UTC m=+21.608094080"
Sep 10 16:02:07.260754 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.260542    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:07.260880 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.260591    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:07.260880 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:07.260837    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:07.260965 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:07.260911    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:07.358948 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.358912    2574 generic.go:356] "Generic (PLEG): container finished" podID="6aefb6f6c14fbe159c01d407c6986b03" containerID="9a0cefa3ea51f12a0b9c5dbba337ae34817ce977aacae4e6b4499c1f5384f2e1" exitCode=0
Sep 10 16:02:07.359517 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.358995    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal" event={"ID":"6aefb6f6c14fbe159c01d407c6986b03","Type":"ContainerDied","Data":"9a0cefa3ea51f12a0b9c5dbba337ae34817ce977aacae4e6b4499c1f5384f2e1"}
Sep 10 16:02:07.361923 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.361907    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:02:07.362196 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362176    2574 generic.go:356] "Generic (PLEG): container finished" podID="a782a69a-5e9e-4eec-9add-50617b03773d" containerID="ab1552386c0dd226c1e992a0f329ff96332b6c2c6002e37dc06eb648ac96a3db" exitCode=1
Sep 10 16:02:07.362261 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362246    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"d8b281a5def8a86bfbd962533cdb0c9a712c153e8d6c5fee18874dd182863290"}
Sep 10 16:02:07.362339 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"c12ea2ff37c874aef3ad58711ac2c88261ead23aa5c3121824c93eff39be25b8"}
Sep 10 16:02:07.362394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"85b2be6872434ff09ab0c91144c808a73caa4387bcc8bce374847802935b000b"}
Sep 10 16:02:07.362430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362409    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"dbcf15ea1d18cfb03895f59e494d0bd58be46b54475ce2f634ae919da6adb759"}
Sep 10 16:02:07.362474 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.362455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerDied","Data":"ab1552386c0dd226c1e992a0f329ff96332b6c2c6002e37dc06eb648ac96a3db"}
Sep 10 16:02:07.363740 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.363718    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-694zm" event={"ID":"96176111-14ab-4cfb-96cc-7884315c2410","Type":"ContainerStarted","Data":"8363fc1bee1bd32d890904941970aced44640520e65a021898270a3a165e4b33"}
Sep 10 16:02:07.365090 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.365059    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7" event={"ID":"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa","Type":"ContainerStarted","Data":"8cc60c4c7929b898460f212f6cbe2a1cdf41f2ecb29836af8b4263e3fbe19431"}
Sep 10 16:02:07.366256 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.366239    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ms7g8" event={"ID":"9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c","Type":"ContainerStarted","Data":"d5421738fe80dee9f9d0b05b10a3156534e183dbecf5f511a01da7ad07e74db2"}
Sep 10 16:02:07.367699 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.367682    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="f0264085e1ab7e49ee00d419311a8eca439e5a2c5f8941742489d3665e4819fa" exitCode=0
Sep 10 16:02:07.367766 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.367740    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"f0264085e1ab7e49ee00d419311a8eca439e5a2c5f8941742489d3665e4819fa"}
Sep 10 16:02:07.369274 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.369248    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vp6vj" event={"ID":"e8ebdd40-30cf-4864-9b3a-1827cbc9d72a","Type":"ContainerStarted","Data":"c6f3e1b337f62e23ff20e96dfca09420920c5489efd7cd020d94342964af54d5"}
Sep 10 16:02:07.412760 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.412697    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-169.ec2.internal" podStartSLOduration=21.412676677 podStartE2EDuration="21.412676677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:07.396082984 +0000 UTC m=+22.629031087" watchObservedRunningTime="2026-09-10 16:02:07.412676677 +0000 UTC m=+22.645624780"
Sep 10 16:02:07.436725 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.436665    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vp6vj" podStartSLOduration=3.728867501 podStartE2EDuration="22.436643426s" totalImagesPullingTime="18.707775925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:07.412815419 +0000 UTC m=+22.645763539" watchObservedRunningTime="2026-09-10 16:02:07.436643426 +0000 UTC m=+22.669591529"
Sep 10 16:02:07.453155 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.453098    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-694zm" podStartSLOduration=3.70561733 podStartE2EDuration="22.453075316s" totalImagesPullingTime="18.747457986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:07.453036925 +0000 UTC m=+22.685985052" watchObservedRunningTime="2026-09-10 16:02:07.453075316 +0000 UTC m=+22.686023418"
Sep 10 16:02:07.469860 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:07.469815    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ms7g8" podStartSLOduration=3.994519506 podStartE2EDuration="22.46979561s" totalImagesPullingTime="18.475276104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:07.469613261 +0000 UTC m=+22.702561363" watchObservedRunningTime="2026-09-10 16:02:07.46979561 +0000 UTC m=+22.702743713"
Sep 10 16:02:08.157248 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.157208    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 16:02:08.225573 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.225478    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T16:02:08.157230282Z","UUID":"130de39d-fd29-4263-a3c8-25f059a51335","Handler":null,"Name":"","Endpoint":""}
Sep 10 16:02:08.227081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.227060    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 16:02:08.227203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.227092    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 16:02:08.260233 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.260204    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:08.260424 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:08.260304    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:08.375828 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.375735    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7" event={"ID":"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa","Type":"ContainerStarted","Data":"bf860628925fc993e3f35d8c14a8bf648acc9fc80efd1179a2d6b64152cc7efa"}
Sep 10 16:02:08.377880 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.377847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal" event={"ID":"6aefb6f6c14fbe159c01d407c6986b03","Type":"ContainerStarted","Data":"93eea5336b84ddd7aefdcdf47e06906b466d13b0ba0b2e9d877843f46f537c6f"}
Sep 10 16:02:08.379548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.379521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8qpkg" event={"ID":"4b9c8e0f-377a-47c3-a3e9-d048a4a703ed","Type":"ContainerStarted","Data":"8a6f28c6bc27d8095829f3f8190413bf8794672cc5c4af0ad1ac72760eb15a82"}
Sep 10 16:02:08.394020 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.393970    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-169.ec2.internal" podStartSLOduration=22.393953964 podStartE2EDuration="22.393953964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:08.393943382 +0000 UTC m=+23.626891485" watchObservedRunningTime="2026-09-10 16:02:08.393953964 +0000 UTC m=+23.626902066"
Sep 10 16:02:08.410079 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:08.410038    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8qpkg" podStartSLOduration=4.725832807 podStartE2EDuration="23.410025931s" totalImagesPullingTime="18.684193124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:08.409418649 +0000 UTC m=+23.642366824" watchObservedRunningTime="2026-09-10 16:02:08.410025931 +0000 UTC m=+23.642974060"
Sep 10 16:02:09.243287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.243192    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:02:09.244078 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.244056    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:02:09.259998 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.259974    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:09.260109 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:09.260088    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:09.260198 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.260158    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:09.260302 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:09.260280    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:09.385457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.385423    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:02:09.385995 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:09.385953    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"9ffb8c745ff642f8a7e20b0c409d3e8eeeea5c44ad0a3d5b36d246f09b5c886e"}
Sep 10 16:02:10.260982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.260747    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:10.261197 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:10.261078    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:10.393579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.393533    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7" event={"ID":"b3a85a5d-4b7e-43f5-a302-d6392e3ac5aa","Type":"ContainerStarted","Data":"6c2f5d54c93887bf66ab2a4f60c4eded9528feea416bb50273b0fc5c8f20bf2c"}
Sep 10 16:02:10.393579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.393571    2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 16:02:10.564749 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.564672    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:02:10.565348 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.565303    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vp6vj"
Sep 10 16:02:10.583486 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:10.583437    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n7gv7" podStartSLOduration=4.374059544 podStartE2EDuration="25.58341644s" totalImagesPullingTime="21.209356896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:10.414058977 +0000 UTC m=+25.647007079" watchObservedRunningTime="2026-09-10 16:02:10.58341644 +0000 UTC m=+25.816364573"
Sep 10 16:02:11.260298 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:11.260232    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:11.260298 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:11.260256    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:11.260475 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:11.260383    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:11.260513 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:11.260477    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:12.260180 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.259986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:12.260548 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:12.260219    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:12.400197 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.399784    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="0e3d959f8b8542614355607e7acc3dce060d28cb899c900047856fa9ebb0b4ac" exitCode=0
Sep 10 16:02:12.400197 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.399881    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"0e3d959f8b8542614355607e7acc3dce060d28cb899c900047856fa9ebb0b4ac"}
Sep 10 16:02:12.404564 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.404544    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:02:12.404940 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.404921    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"bb1e0a567bd90801fb3e7b3e6134e1a24019709f81a7c643937eb6fcdd352878"}
Sep 10 16:02:12.405181 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.405162    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:12.405276 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.405233    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:12.405350 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.405285    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:12.405350 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.405339    2574 scope.go:117] "RemoveContainer" containerID="ab1552386c0dd226c1e992a0f329ff96332b6c2c6002e37dc06eb648ac96a3db"
Sep 10 16:02:12.420728 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.420707    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:12.420991 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:12.420975    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:13.260088 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.260051    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:13.260429 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:13.260177    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:13.260429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.260230    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:13.260429 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:13.260344    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:13.411051 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.411026    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:02:13.411436 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.411411    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" event={"ID":"a782a69a-5e9e-4eec-9add-50617b03773d","Type":"ContainerStarted","Data":"c33d4abd9da539a7f37bc43afb2ca2b49746d3e3d2e6c33f2a41a9e5eaa4a669"}
Sep 10 16:02:13.440449 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.440391    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr" podStartSLOduration=8.77043345 podStartE2EDuration="27.440367487s" totalImagesPullingTime="18.669934037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:13.440234325 +0000 UTC m=+28.673182427" watchObservedRunningTime="2026-09-10 16:02:13.440367487 +0000 UTC m=+28.673315590"
Sep 10 16:02:13.913549 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.913514    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c7krj"]
Sep 10 16:02:13.913677 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.913660    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:13.913798 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:13.913775    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:13.917094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.917066    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lns56"]
Sep 10 16:02:13.917239 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.917190    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:13.917381 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:13.917310    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:13.917890 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.917867    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c67tz"]
Sep 10 16:02:13.917996 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:13.917955    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:13.918073 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:13.918050    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:14.415046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:14.414963    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="990a17990b6f2bf824f24d23a0935a1e85994c292b41ce178b58c987d80522cc" exitCode=0
Sep 10 16:02:14.415721 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:14.415046    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"990a17990b6f2bf824f24d23a0935a1e85994c292b41ce178b58c987d80522cc"}
Sep 10 16:02:15.261985 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:15.261953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:15.262185 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:15.262072    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:16.260849 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:16.260679    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:16.261226 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:16.260724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:16.261226 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:16.260929    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:16.261226 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:16.260983    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:16.421925 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:16.421846    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="78c93e07106af21482d8422851fd0060b453269c1741d0112cd9345d1a9e97c5" exitCode=0
Sep 10 16:02:16.421925 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:16.421899    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"78c93e07106af21482d8422851fd0060b453269c1741d0112cd9345d1a9e97c5"}
Sep 10 16:02:17.248353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.248268    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z8cg7"]
Sep 10 16:02:17.255620 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.255592    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.258307 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.258282    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 16:02:17.258656 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.258630    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-86fxw\""
Sep 10 16:02:17.259592 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.259568    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 16:02:17.260572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.260559    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:17.260669 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:17.260651    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c7krj" podUID="e130d30f-5fe3-4bcd-825b-c52b6b9a6e27"
Sep 10 16:02:17.413981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.413950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-hosts-file\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.414459 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.414005    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmjw\" (UniqueName: \"kubernetes.io/projected/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-kube-api-access-qcmjw\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.414459 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.414035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-tmp-dir\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.514949 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.514866    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-hosts-file\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.514949 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.514928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmjw\" (UniqueName: \"kubernetes.io/projected/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-kube-api-access-qcmjw\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.515124 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.514954    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-tmp-dir\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.515124 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.515054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-hosts-file\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.515398 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.515380    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-tmp-dir\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.532776 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.532748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmjw\" (UniqueName: \"kubernetes.io/projected/c2ed7b68-a01f-4353-a4ed-e0faeef067ad-kube-api-access-qcmjw\") pod \"node-resolver-z8cg7\" (UID: \"c2ed7b68-a01f-4353-a4ed-e0faeef067ad\") " pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.564570 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:17.564545    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z8cg7"
Sep 10 16:02:17.573518 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:17.573494    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ed7b68_a01f_4353_a4ed_e0faeef067ad.slice/crio-fe3ca728d98a6792621dd56eb813e069d353995226ce9c31a3be2b4f3431019a WatchSource:0}: Error finding container fe3ca728d98a6792621dd56eb813e069d353995226ce9c31a3be2b4f3431019a: Status 404 returned error can't find the container with id fe3ca728d98a6792621dd56eb813e069d353995226ce9c31a3be2b4f3431019a
Sep 10 16:02:18.261028 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:18.260836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:18.261284 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:18.260898    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:18.261284 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:18.261118    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c67tz" podUID="0f145856-9b25-42b2-b0cc-3c6de1a7ef1c"
Sep 10 16:02:18.261284 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:18.261230    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lns56" podUID="d1e89918-a035-4d0c-9100-2db3b17e4310"
Sep 10 16:02:18.428891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:18.428852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z8cg7" event={"ID":"c2ed7b68-a01f-4353-a4ed-e0faeef067ad","Type":"ContainerStarted","Data":"2b0cbd1233d7f4d095977139253b5896d5d8ae9aea40c1ad89d4036f9793d4f0"}
Sep 10 16:02:18.429382 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:18.428901    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z8cg7" event={"ID":"c2ed7b68-a01f-4353-a4ed-e0faeef067ad","Type":"ContainerStarted","Data":"fe3ca728d98a6792621dd56eb813e069d353995226ce9c31a3be2b4f3431019a"}
Sep 10 16:02:18.452563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:18.452521    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z8cg7" podStartSLOduration=1.452504422 podStartE2EDuration="1.452504422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:18.452191852 +0000 UTC m=+33.685139957" watchObservedRunningTime="2026-09-10 16:02:18.452504422 +0000 UTC m=+33.685452526"
Sep 10 16:02:19.025431 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.025399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:19.025602 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.025446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:19.025602 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.025467    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:19.025602 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025559    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:02:19.025602 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025575    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:02:19.025602 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025597    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:02:19.025794 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025609    2574 projected.go:196] Error preparing data for projected volume kube-api-access-rkl7m for pod openshift-network-diagnostics/network-check-target-c67tz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:02:19.025794 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025632    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs podName:d1e89918-a035-4d0c-9100-2db3b17e4310 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:51.025617566 +0000 UTC m=+66.258565646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs") pod "network-metrics-daemon-lns56" (UID: "d1e89918-a035-4d0c-9100-2db3b17e4310") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:02:19.025794 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025561    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:02:19.025794 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025650    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m podName:0f145856-9b25-42b2-b0cc-3c6de1a7ef1c nodeName:}" failed. No retries permitted until 2026-09-10 16:02:51.025639294 +0000 UTC m=+66.258587374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rkl7m" (UniqueName: "kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m") pod "network-check-target-c67tz" (UID: "0f145856-9b25-42b2-b0cc-3c6de1a7ef1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:02:19.025794 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.025680    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret podName:e130d30f-5fe3-4bcd-825b-c52b6b9a6e27 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:51.025670092 +0000 UTC m=+66.258618178 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret") pod "global-pull-secret-syncer-c7krj" (UID: "e130d30f-5fe3-4bcd-825b-c52b6b9a6e27") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:02:19.083569 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.083531    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-169.ec2.internal" event="NodeReady"
Sep 10 16:02:19.083723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.083681    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 16:02:19.139838 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.139756    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z4t22"]
Sep 10 16:02:19.141725 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.141704    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v7njq"]
Sep 10 16:02:19.141868 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.141846    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.144195 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.144172    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.144832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.144517    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fqpr5\""
Sep 10 16:02:19.144832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.144732    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 16:02:19.144981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.144961    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 16:02:19.147048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.146573    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 16:02:19.147048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.146940    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 16:02:19.147048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.146982    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 16:02:19.147259 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.147139    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-twhfc\""
Sep 10 16:02:19.154660 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.154596    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v7njq"]
Sep 10 16:02:19.155300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.155282    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z4t22"]
Sep 10 16:02:19.226726 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdlgd\" (UniqueName: \"kubernetes.io/projected/4b3b5327-2865-4fdf-9b5a-be0f85204661-kube-api-access-wdlgd\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.226726 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3b5327-2865-4fdf-9b5a-be0f85204661-config-volume\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.226914 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52ec76f1-3c0f-4241-b6b5-03eb131577e4-cert\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.226914 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pps2k\" (UniqueName: \"kubernetes.io/projected/52ec76f1-3c0f-4241-b6b5-03eb131577e4-kube-api-access-pps2k\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.226914 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3b5327-2865-4fdf-9b5a-be0f85204661-tmp-dir\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.227024 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.226922    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.260232 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.260207    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:19.263345 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.263303    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 16:02:19.327786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdlgd\" (UniqueName: \"kubernetes.io/projected/4b3b5327-2865-4fdf-9b5a-be0f85204661-kube-api-access-wdlgd\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.327786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3b5327-2865-4fdf-9b5a-be0f85204661-config-volume\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.328023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52ec76f1-3c0f-4241-b6b5-03eb131577e4-cert\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.328023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pps2k\" (UniqueName: \"kubernetes.io/projected/52ec76f1-3c0f-4241-b6b5-03eb131577e4-kube-api-access-pps2k\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.328023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327849    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3b5327-2865-4fdf-9b5a-be0f85204661-tmp-dir\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.328023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.327903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.328239 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.328027    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:19.328239 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.328093    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:19.828075762 +0000 UTC m=+35.061023858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:19.328422 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.328313    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b3b5327-2865-4fdf-9b5a-be0f85204661-tmp-dir\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.328422 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.328381    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b3b5327-2865-4fdf-9b5a-be0f85204661-config-volume\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.331377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.331350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52ec76f1-3c0f-4241-b6b5-03eb131577e4-cert\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.338033 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.338000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdlgd\" (UniqueName: \"kubernetes.io/projected/4b3b5327-2865-4fdf-9b5a-be0f85204661-kube-api-access-wdlgd\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.338186 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.338168    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pps2k\" (UniqueName: \"kubernetes.io/projected/52ec76f1-3c0f-4241-b6b5-03eb131577e4-kube-api-access-pps2k\") pod \"ingress-canary-v7njq\" (UID: \"52ec76f1-3c0f-4241-b6b5-03eb131577e4\") " pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.462723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.462650    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v7njq"
Sep 10 16:02:19.831744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:19.831655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:19.831880 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.831769    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:19.831880 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:19.831841    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:20.83182496 +0000 UTC m=+36.064773044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:20.240013 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.239934    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"]
Sep 10 16:02:20.241933 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.241912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.244671 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.244642    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 16:02:20.244792 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.244695    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:02:20.245023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.245000    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 16:02:20.246161 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.246141    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 16:02:20.246252 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.246145    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pphw4\""
Sep 10 16:02:20.251999 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.251980    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"]
Sep 10 16:02:20.260930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.260908    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:20.261109 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.261085    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:20.264021 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.264001    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 16:02:20.264120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.264003    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 16:02:20.264120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.264056    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5gt7\""
Sep 10 16:02:20.264213 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.264189    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-swxfx\""
Sep 10 16:02:20.264213 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.264194    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 16:02:20.334760 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.334729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.334921 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.334797    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.334921 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.334822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq77p\" (UniqueName: \"kubernetes.io/projected/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-kube-api-access-kq77p\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.435811 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.435775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.435999 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.435880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.435999 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.435909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq77p\" (UniqueName: \"kubernetes.io/projected/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-kube-api-access-kq77p\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.436471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.436450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.438148 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.438127    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.453937 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.453911    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq77p\" (UniqueName: \"kubernetes.io/projected/1934703d-0bf7-4a49-a6dd-9e79bf4d0806-kube-api-access-kq77p\") pod \"kube-storage-version-migrator-operator-8d7dd4675-f8jqd\" (UID: \"1934703d-0bf7-4a49-a6dd-9e79bf4d0806\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.552697 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.552615    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"
Sep 10 16:02:20.839382 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:20.839220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:20.839622 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:20.839396    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:20.839622 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:20.839480    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:22.839458622 +0000 UTC m=+38.072406707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:21.080693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.080496    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v7njq"]
Sep 10 16:02:21.083491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.083463    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd"]
Sep 10 16:02:21.086621 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:21.086591    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ec76f1_3c0f_4241_b6b5_03eb131577e4.slice/crio-f5fc82d68003ed2e34528a5d6aad3beef6fc2395f8ebaa6f4a879a532635412f WatchSource:0}: Error finding container f5fc82d68003ed2e34528a5d6aad3beef6fc2395f8ebaa6f4a879a532635412f: Status 404 returned error can't find the container with id f5fc82d68003ed2e34528a5d6aad3beef6fc2395f8ebaa6f4a879a532635412f
Sep 10 16:02:21.087312 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:21.087283    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1934703d_0bf7_4a49_a6dd_9e79bf4d0806.slice/crio-1538a46fd7d05b6a17458f34878fe34c530f28bee9281910a2bf57efa012adbc WatchSource:0}: Error finding container 1538a46fd7d05b6a17458f34878fe34c530f28bee9281910a2bf57efa012adbc: Status 404 returned error can't find the container with id 1538a46fd7d05b6a17458f34878fe34c530f28bee9281910a2bf57efa012adbc
Sep 10 16:02:21.232860 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.232824    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"]
Sep 10 16:02:21.235518 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.235499    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.238132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.238112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 16:02:21.238220 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.238114    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 16:02:21.238535 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.238515    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t5qj5\""
Sep 10 16:02:21.238620 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.238552    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 16:02:21.238620 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.238572    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:02:21.242631 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.242610    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"]
Sep 10 16:02:21.342628 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.342567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b8f7029-b8bd-469a-8019-72c207ff1b9a-serving-cert\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.342628 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.342592    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b8f7029-b8bd-469a-8019-72c207ff1b9a-config\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.342762 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.342644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4rlg\" (UniqueName: \"kubernetes.io/projected/9b8f7029-b8bd-469a-8019-72c207ff1b9a-kube-api-access-z4rlg\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.438639 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.438608    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="3e13054bce4e1fc1ef5f7fb0091a098ad40f9b7397495b25b589c187fce71875" exitCode=0
Sep 10 16:02:21.438742 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.438698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"3e13054bce4e1fc1ef5f7fb0091a098ad40f9b7397495b25b589c187fce71875"}
Sep 10 16:02:21.439891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.439865    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd" event={"ID":"1934703d-0bf7-4a49-a6dd-9e79bf4d0806","Type":"ContainerStarted","Data":"1538a46fd7d05b6a17458f34878fe34c530f28bee9281910a2bf57efa012adbc"}
Sep 10 16:02:21.441115 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.441081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v7njq" event={"ID":"52ec76f1-3c0f-4241-b6b5-03eb131577e4","Type":"ContainerStarted","Data":"f5fc82d68003ed2e34528a5d6aad3beef6fc2395f8ebaa6f4a879a532635412f"}
Sep 10 16:02:21.443407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.443382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b8f7029-b8bd-469a-8019-72c207ff1b9a-serving-cert\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.443407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.443416    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b8f7029-b8bd-469a-8019-72c207ff1b9a-config\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.443575 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.443477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rlg\" (UniqueName: \"kubernetes.io/projected/9b8f7029-b8bd-469a-8019-72c207ff1b9a-kube-api-access-z4rlg\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.444145 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.444125    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b8f7029-b8bd-469a-8019-72c207ff1b9a-config\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.446661 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.446547    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b8f7029-b8bd-469a-8019-72c207ff1b9a-serving-cert\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.452475 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.452452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4rlg\" (UniqueName: \"kubernetes.io/projected/9b8f7029-b8bd-469a-8019-72c207ff1b9a-kube-api-access-z4rlg\") pod \"service-ca-operator-859f94c4b-n22kx\" (UID: \"9b8f7029-b8bd-469a-8019-72c207ff1b9a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.544080 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.544048    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"
Sep 10 16:02:21.821123 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:21.821030    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx"]
Sep 10 16:02:21.825828 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:21.825795    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8f7029_b8bd_469a_8019_72c207ff1b9a.slice/crio-964cd85b698e688ce88abc306bd0611c4af85d43ceca6b5e473a14a33a368fff WatchSource:0}: Error finding container 964cd85b698e688ce88abc306bd0611c4af85d43ceca6b5e473a14a33a368fff: Status 404 returned error can't find the container with id 964cd85b698e688ce88abc306bd0611c4af85d43ceca6b5e473a14a33a368fff
Sep 10 16:02:22.448817 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.448772    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae9df9b4-3da2-464d-a95b-50b09e43ba8d" containerID="ca984e1d58e24a785cb763a31a74de85a45c74f6b6f78b7dbfbf7e44d2020e97" exitCode=0
Sep 10 16:02:22.448964 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.448870    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerDied","Data":"ca984e1d58e24a785cb763a31a74de85a45c74f6b6f78b7dbfbf7e44d2020e97"}
Sep 10 16:02:22.450298 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.450272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx" event={"ID":"9b8f7029-b8bd-469a-8019-72c207ff1b9a","Type":"ContainerStarted","Data":"964cd85b698e688ce88abc306bd0611c4af85d43ceca6b5e473a14a33a368fff"}
Sep 10 16:02:22.795266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.795119    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wc5hh"]
Sep 10 16:02:22.828319 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.828285    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wc5hh"]
Sep 10 16:02:22.828774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.828428    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh"
Sep 10 16:02:22.831545 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.831520    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qkbll\""
Sep 10 16:02:22.854193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.854068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:22.854193 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:22.854186    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:22.854454 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:22.854248    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:26.854229454 +0000 UTC m=+42.087177550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:22.954839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:22.954772    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt7g6\" (UniqueName: \"kubernetes.io/projected/812dea91-7c10-47ab-a38a-6450f6cfea42-kube-api-access-pt7g6\") pod \"network-check-source-685456655f-wc5hh\" (UID: \"812dea91-7c10-47ab-a38a-6450f6cfea42\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh"
Sep 10 16:02:23.055469 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.055390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt7g6\" (UniqueName: \"kubernetes.io/projected/812dea91-7c10-47ab-a38a-6450f6cfea42-kube-api-access-pt7g6\") pod \"network-check-source-685456655f-wc5hh\" (UID: \"812dea91-7c10-47ab-a38a-6450f6cfea42\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh"
Sep 10 16:02:23.067961 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.067929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt7g6\" (UniqueName: \"kubernetes.io/projected/812dea91-7c10-47ab-a38a-6450f6cfea42-kube-api-access-pt7g6\") pod \"network-check-source-685456655f-wc5hh\" (UID: \"812dea91-7c10-47ab-a38a-6450f6cfea42\") " pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh"
Sep 10 16:02:23.141863 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.141825    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh"
Sep 10 16:02:23.247698 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.247663    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cb5f9f868-q8lx9"]
Sep 10 16:02:23.266809 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.266772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.269893 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.269864    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 16:02:23.270776 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.270752    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 16:02:23.270992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.270972    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 16:02:23.271193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.271178    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nqhnk\""
Sep 10 16:02:23.271593 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.271569    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cb5f9f868-q8lx9"]
Sep 10 16:02:23.276267 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.276245    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 16:02:23.279685 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.279662    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k2tkb"]
Sep 10 16:02:23.304287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.304249    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2tkb"]
Sep 10 16:02:23.304464 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.304426    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.307629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.307349    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 16:02:23.307629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.307393    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 16:02:23.307629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.307395    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6s77g\""
Sep 10 16:02:23.307629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.307354    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 16:02:23.307629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.307399    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 16:02:23.357243 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357419 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357419 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357520 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357442    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357564    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwpzt\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.357683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.357647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458436 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458436 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458436 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458615 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.458442    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:23.458615 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.458461    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:23.458615 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.458523    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:23.958507538 +0000 UTC m=+39.191455618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:23.458615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-crio-socket\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458655    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-data-volume\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458674    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwpzt\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt8xf\" (UniqueName: \"kubernetes.io/projected/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-api-access-lt8xf\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.458915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.458784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.459374 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.459351    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.459559 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.459538    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.459977 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.459920    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.461758 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.461719    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.462417 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.462376    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.470600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.470553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.471029 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.470999    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwpzt\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-crio-socket\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-data-volume\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.559377    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.559463    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:24.059442021 +0000 UTC m=+39.292390119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : configmap "kube-rbac-proxy" not found
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.559481    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559520    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt8xf\" (UniqueName: \"kubernetes.io/projected/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-api-access-lt8xf\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.559532    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:24.059515725 +0000 UTC m=+39.292463822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-crio-socket\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.559850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.559762    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-data-volume\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.570075 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.570037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt8xf\" (UniqueName: \"kubernetes.io/projected/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-api-access-lt8xf\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:23.963302 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:23.963221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:23.963705 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.963372    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:23.963705 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.963392    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:23.963705 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:23.963450    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:24.963435217 +0000 UTC m=+40.196383298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:24.064400 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:24.064358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:24.064584 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:24.064438    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:24.064584 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.064544    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:24.064584 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.064545    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:02:24.064748 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.064616    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:25.064594825 +0000 UTC m=+40.297543171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:24.064748 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.064636    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:25.064626264 +0000 UTC m=+40.297574349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : configmap "kube-rbac-proxy" not found
Sep 10 16:02:24.460349 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:24.460293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" event={"ID":"ae9df9b4-3da2-464d-a95b-50b09e43ba8d","Type":"ContainerStarted","Data":"1900e2f132080c01e72034bfdd31388add2e3dcaaac66cdbcdcb2c87e3f869c7"}
Sep 10 16:02:24.488929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:24.486734    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sx9sp" podStartSLOduration=9.280024715 podStartE2EDuration="39.486710133s" totalImagesPullingTime="30.206685418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:24.484539155 +0000 UTC m=+39.717487273" watchObservedRunningTime="2026-09-10 16:02:24.486710133 +0000 UTC m=+39.719658238"
Sep 10 16:02:24.973261 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:24.973232    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:24.973647 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.973424    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:24.973647 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.973444    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:24.973647 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:24.973556    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:26.973533828 +0000 UTC m=+42.206481928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:25.009442 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.009417    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z8cg7_c2ed7b68-a01f-4353-a4ed-e0faeef067ad/dns-node-resolver/0.log"
Sep 10 16:02:25.073837 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.073811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:25.073948 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.073890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:25.074011 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:25.073944    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:02:25.074011 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:25.073986    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:25.074080 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:25.074019    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:27.073990533 +0000 UTC m=+42.306938612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : configmap "kube-rbac-proxy" not found
Sep 10 16:02:25.074080 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:25.074032    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:27.074026572 +0000 UTC m=+42.306974651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:25.237090 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.237007    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-wc5hh"]
Sep 10 16:02:25.304823 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:25.304782    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812dea91_7c10_47ab_a38a_6450f6cfea42.slice/crio-2dea97ea8931fe1782262437ef5b637d2693dfcc54a247902ee09886e67c0c80 WatchSource:0}: Error finding container 2dea97ea8931fe1782262437ef5b637d2693dfcc54a247902ee09886e67c0c80: Status 404 returned error can't find the container with id 2dea97ea8931fe1782262437ef5b637d2693dfcc54a247902ee09886e67c0c80
Sep 10 16:02:25.465128 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.464830    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v7njq" event={"ID":"52ec76f1-3c0f-4241-b6b5-03eb131577e4","Type":"ContainerStarted","Data":"22ed1787b4e7ca986afa7f4e6edce48303397bab11c68babe782b0357e7c49d5"}
Sep 10 16:02:25.466295 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.466264    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh" event={"ID":"812dea91-7c10-47ab-a38a-6450f6cfea42","Type":"ContainerStarted","Data":"2dea97ea8931fe1782262437ef5b637d2693dfcc54a247902ee09886e67c0c80"}
Sep 10 16:02:25.467960 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.467935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd" event={"ID":"1934703d-0bf7-4a49-a6dd-9e79bf4d0806","Type":"ContainerStarted","Data":"b9b74b85ef5558932f6c5deb02cd0c8ab0f5c74c941c99e6fbb6c1e44a603f28"}
Sep 10 16:02:25.469641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.469613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx" event={"ID":"9b8f7029-b8bd-469a-8019-72c207ff1b9a","Type":"ContainerStarted","Data":"dad9f20ea889ce541c82e3f14ef26d4e48a70eecde433eda03243847ec5a752e"}
Sep 10 16:02:25.485056 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.485001    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v7njq" podStartSLOduration=2.605967081 podStartE2EDuration="6.484983949s" totalImagesPullingTime="3.879016868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:25.48423337 +0000 UTC m=+40.717181473" watchObservedRunningTime="2026-09-10 16:02:25.484983949 +0000 UTC m=+40.717932087"
Sep 10 16:02:25.503750 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.503642    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx" podStartSLOduration=0.994274349 podStartE2EDuration="4.503619321s" totalImagesPullingTime="3.509344972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:25.503352598 +0000 UTC m=+40.736300695" watchObservedRunningTime="2026-09-10 16:02:25.503619321 +0000 UTC m=+40.736567441"
Sep 10 16:02:25.521395 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.521345    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd" podStartSLOduration=1.653363717 podStartE2EDuration="5.52130876s" totalImagesPullingTime="3.867945043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:25.521295367 +0000 UTC m=+40.754243479" watchObservedRunningTime="2026-09-10 16:02:25.52130876 +0000 UTC m=+40.754256855"
Sep 10 16:02:25.815956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:25.815884    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ms7g8_9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c/node-ca/0.log"
Sep 10 16:02:26.209767 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.209679    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"]
Sep 10 16:02:26.226355 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.226168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"
Sep 10 16:02:26.228120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.228089    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"]
Sep 10 16:02:26.228687 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.228664    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 16:02:26.229177 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.229155    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 16:02:26.230242 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.230052    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zrbkx\""
Sep 10 16:02:26.384461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.384426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqwn5\" (UniqueName: \"kubernetes.io/projected/6573b778-63d7-4fe9-bfe8-f65d5d33db27-kube-api-access-bqwn5\") pod \"migrator-fdbb8fdff-7ztdl\" (UID: \"6573b778-63d7-4fe9-bfe8-f65d5d33db27\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"
Sep 10 16:02:26.484993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.484833    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqwn5\" (UniqueName: \"kubernetes.io/projected/6573b778-63d7-4fe9-bfe8-f65d5d33db27-kube-api-access-bqwn5\") pod \"migrator-fdbb8fdff-7ztdl\" (UID: \"6573b778-63d7-4fe9-bfe8-f65d5d33db27\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"
Sep 10 16:02:26.496760 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.496731    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqwn5\" (UniqueName: \"kubernetes.io/projected/6573b778-63d7-4fe9-bfe8-f65d5d33db27-kube-api-access-bqwn5\") pod \"migrator-fdbb8fdff-7ztdl\" (UID: \"6573b778-63d7-4fe9-bfe8-f65d5d33db27\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"
Sep 10 16:02:26.538710 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.538682    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"
Sep 10 16:02:26.609577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.609545    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v7njq_52ec76f1-3c0f-4241-b6b5-03eb131577e4/serve-healthcheck-canary/0.log"
Sep 10 16:02:26.840908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.840830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl"]
Sep 10 16:02:26.845665 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:26.845629    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6573b778_63d7_4fe9_bfe8_f65d5d33db27.slice/crio-437fe8ae97c13735afea79e7da936d7adde36e1f21e5bfdcc6bb8fb2cd7ed04c WatchSource:0}: Error finding container 437fe8ae97c13735afea79e7da936d7adde36e1f21e5bfdcc6bb8fb2cd7ed04c: Status 404 returned error can't find the container with id 437fe8ae97c13735afea79e7da936d7adde36e1f21e5bfdcc6bb8fb2cd7ed04c
Sep 10 16:02:26.888567 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.888536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:26.888767 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:26.888737    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:26.888872 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:26.888816    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:34.888795526 +0000 UTC m=+50.121743619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:26.989300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:26.989257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:26.989560 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:26.989537    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:26.989560 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:26.989562    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:26.989716 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:26.989632    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:30.98960779 +0000 UTC m=+46.222555894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:27.090055 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:27.090009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:27.090237 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:27.090095    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:27.090237 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:27.090153    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:27.090237 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:27.090190    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:02:27.090414 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:27.090240    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:31.090219775 +0000 UTC m=+46.323167863 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:27.090414 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:27.090260    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:31.090251272 +0000 UTC m=+46.323199354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : configmap "kube-rbac-proxy" not found
Sep 10 16:02:27.477204 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:27.477168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl" event={"ID":"6573b778-63d7-4fe9-bfe8-f65d5d33db27","Type":"ContainerStarted","Data":"437fe8ae97c13735afea79e7da936d7adde36e1f21e5bfdcc6bb8fb2cd7ed04c"}
Sep 10 16:02:29.485052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:29.484957    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl" event={"ID":"6573b778-63d7-4fe9-bfe8-f65d5d33db27","Type":"ContainerStarted","Data":"b9f37826c013892a65d78852111c278c50a8d33a003344a06ded8e7591f08bf9"}
Sep 10 16:02:29.485052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:29.485023    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl" event={"ID":"6573b778-63d7-4fe9-bfe8-f65d5d33db27","Type":"ContainerStarted","Data":"1d3b1909717446829214d04ed22c75bc3fc29e9908ae75df8207ea2dcdab4cf8"}
Sep 10 16:02:29.486472 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:29.486448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh" event={"ID":"812dea91-7c10-47ab-a38a-6450f6cfea42","Type":"ContainerStarted","Data":"d6e389df29223f44d85ca568539aac385be6ac35bf4dcfb5dd59037a1cc8ce24"}
Sep 10 16:02:29.545066 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:29.545010    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-7ztdl" podStartSLOduration=1.316217846 podStartE2EDuration="3.544991804s" totalImagesPullingTime="2.228773958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:29.5177006 +0000 UTC m=+44.750648701" watchObservedRunningTime="2026-09-10 16:02:29.544991804 +0000 UTC m=+44.777939905"
Sep 10 16:02:29.545242 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:29.545188    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-wc5hh" podStartSLOduration=3.80047657 podStartE2EDuration="7.545182201s" totalImagesPullingTime="3.744705631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:29.544932436 +0000 UTC m=+44.777880538" watchObservedRunningTime="2026-09-10 16:02:29.545182201 +0000 UTC m=+44.778130338"
Sep 10 16:02:31.024805 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:31.024760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:31.025188 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.024950    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:31.025188 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.024974    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:31.025188 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.025033    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:39.025017041 +0000 UTC m=+54.257965125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:31.125588 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:31.125550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:31.125706 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:31.125612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:31.125742 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.125704    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:31.125774 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.125769    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:39.125754028 +0000 UTC m=+54.358702107 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:31.125825 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.125712    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:02:31.125866 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:31.125836    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:39.125823535 +0000 UTC m=+54.358771615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : configmap "kube-rbac-proxy" not found
Sep 10 16:02:34.956500 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:34.956460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:34.957008 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:34.956645    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:02:34.957008 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:34.956734    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls podName:4b3b5327-2865-4fdf-9b5a-be0f85204661 nodeName:}" failed. No retries permitted until 2026-09-10 16:02:50.956710479 +0000 UTC m=+66.189658561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls") pod "dns-default-z4t22" (UID: "4b3b5327-2865-4fdf-9b5a-be0f85204661") : secret "dns-default-metrics-tls" not found
Sep 10 16:02:39.083744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:39.083709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") pod \"image-registry-5cb5f9f868-q8lx9\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") " pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:39.084298 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:39.083893    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:02:39.084298 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:39.083915    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb5f9f868-q8lx9: secret "image-registry-tls" not found
Sep 10 16:02:39.084298 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:39.083984    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls podName:70e1937b-6dec-4002-9ccf-d1f7ea622f6a nodeName:}" failed. No retries permitted until 2026-09-10 16:02:55.083963262 +0000 UTC m=+70.316911342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls") pod "image-registry-5cb5f9f868-q8lx9" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a") : secret "image-registry-tls" not found
Sep 10 16:02:39.184404 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:39.184375    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:39.184557 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:39.184484    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:39.184644 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:39.184624    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:39.184696 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:39.184685    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls podName:8d3ddef0-fd8c-4346-8b70-7cad9d7362bb nodeName:}" failed. No retries permitted until 2026-09-10 16:02:55.184672416 +0000 UTC m=+70.417620508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k2tkb" (UID: "8d3ddef0-fd8c-4346-8b70-7cad9d7362bb") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:02:39.184876 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:39.184858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:44.429629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:44.429597    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pskgr"
Sep 10 16:02:50.972230 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:50.972193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:50.974473 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:50.974452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b3b5327-2865-4fdf-9b5a-be0f85204661-metrics-tls\") pod \"dns-default-z4t22\" (UID: \"4b3b5327-2865-4fdf-9b5a-be0f85204661\") " pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:51.072896 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.072857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:51.072896 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.072898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:51.073116 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.072933    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:51.075439 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.075417    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkl7m\" (UniqueName: \"kubernetes.io/projected/0f145856-9b25-42b2-b0cc-3c6de1a7ef1c-kube-api-access-rkl7m\") pod \"network-check-target-c67tz\" (UID: \"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c\") " pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:51.075690 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.075671    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 16:02:51.075774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.075731    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 16:02:51.086078 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.086048    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1e89918-a035-4d0c-9100-2db3b17e4310-metrics-certs\") pod \"network-metrics-daemon-lns56\" (UID: \"d1e89918-a035-4d0c-9100-2db3b17e4310\") " pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:51.086078 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.086065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e130d30f-5fe3-4bcd-825b-c52b6b9a6e27-original-pull-secret\") pod \"global-pull-secret-syncer-c7krj\" (UID: \"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27\") " pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:51.174339 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.174308    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g5gt7\""
Sep 10 16:02:51.179231 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.179215    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-swxfx\""
Sep 10 16:02:51.182376 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.182363    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:51.187596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.187579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lns56"
Sep 10 16:02:51.259096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.258942    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fqpr5\""
Sep 10 16:02:51.267318 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.266503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:51.370769 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.370740    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c7krj"
Sep 10 16:02:51.462797 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.462770    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c67tz"]
Sep 10 16:02:51.467243 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:51.467210    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f145856_9b25_42b2_b0cc_3c6de1a7ef1c.slice/crio-89a24144b7521258d0b97fe6aa33c0c37279a3a6fe7fdc61c71aa05238435966 WatchSource:0}: Error finding container 89a24144b7521258d0b97fe6aa33c0c37279a3a6fe7fdc61c71aa05238435966: Status 404 returned error can't find the container with id 89a24144b7521258d0b97fe6aa33c0c37279a3a6fe7fdc61c71aa05238435966
Sep 10 16:02:51.479764 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.479740    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lns56"]
Sep 10 16:02:51.482246 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:51.482228    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1e89918_a035_4d0c_9100_2db3b17e4310.slice/crio-cbc738fbaf8a5b63975dd143731457a11d19f0a5dd8b1caf5215c771da0ca08f WatchSource:0}: Error finding container cbc738fbaf8a5b63975dd143731457a11d19f0a5dd8b1caf5215c771da0ca08f: Status 404 returned error can't find the container with id cbc738fbaf8a5b63975dd143731457a11d19f0a5dd8b1caf5215c771da0ca08f
Sep 10 16:02:51.549929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.549900    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z4t22"]
Sep 10 16:02:51.553485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.553454    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c67tz" event={"ID":"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c","Type":"ContainerStarted","Data":"89a24144b7521258d0b97fe6aa33c0c37279a3a6fe7fdc61c71aa05238435966"}
Sep 10 16:02:51.553704 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:51.553682    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3b5327_2865_4fdf_9b5a_be0f85204661.slice/crio-f8cf0fd8bd5d45f76bb404d60a82b4d4b0d06ff8d49e550ac303ac61887c3622 WatchSource:0}: Error finding container f8cf0fd8bd5d45f76bb404d60a82b4d4b0d06ff8d49e550ac303ac61887c3622: Status 404 returned error can't find the container with id f8cf0fd8bd5d45f76bb404d60a82b4d4b0d06ff8d49e550ac303ac61887c3622
Sep 10 16:02:51.554739 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.554711    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lns56" event={"ID":"d1e89918-a035-4d0c-9100-2db3b17e4310","Type":"ContainerStarted","Data":"cbc738fbaf8a5b63975dd143731457a11d19f0a5dd8b1caf5215c771da0ca08f"}
Sep 10 16:02:51.663078 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:51.663050    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c7krj"]
Sep 10 16:02:51.666143 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:51.666119    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode130d30f_5fe3_4bcd_825b_c52b6b9a6e27.slice/crio-9dc9b9eb7bfabb9734c317d064a9f00079c9c15d488ed696320479dbd733a4cc WatchSource:0}: Error finding container 9dc9b9eb7bfabb9734c317d064a9f00079c9c15d488ed696320479dbd733a4cc: Status 404 returned error can't find the container with id 9dc9b9eb7bfabb9734c317d064a9f00079c9c15d488ed696320479dbd733a4cc
Sep 10 16:02:52.561477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:52.561432    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c7krj" event={"ID":"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27","Type":"ContainerStarted","Data":"9dc9b9eb7bfabb9734c317d064a9f00079c9c15d488ed696320479dbd733a4cc"}
Sep 10 16:02:52.563688 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:52.563625    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z4t22" event={"ID":"4b3b5327-2865-4fdf-9b5a-be0f85204661","Type":"ContainerStarted","Data":"f8cf0fd8bd5d45f76bb404d60a82b4d4b0d06ff8d49e550ac303ac61887c3622"}
Sep 10 16:02:52.565766 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:52.565710    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c67tz" event={"ID":"0f145856-9b25-42b2-b0cc-3c6de1a7ef1c","Type":"ContainerStarted","Data":"d3c516ce6cc34b8a4d2f487a71021552aecd38d997eb711b1dc0fc8f3f4c9221"}
Sep 10 16:02:52.566064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:52.566002    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:02:52.585568 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:52.584952    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c67tz" podStartSLOduration=67.58493407 podStartE2EDuration="1m7.58493407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:52.584916857 +0000 UTC m=+67.817864960" watchObservedRunningTime="2026-09-10 16:02:52.58493407 +0000 UTC m=+67.817882174"
Sep 10 16:02:53.423164 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.423128    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"]
Sep 10 16:02:53.439265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.439241    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.441835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.441811    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 16:02:53.442011 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.441986    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 16:02:53.443075 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.443041    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 16:02:53.443216 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.443073    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 16:02:53.443216 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.443195    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 16:02:53.443364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.443319    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 16:02:53.443430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.443415    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 16:02:53.461283 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.461257    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"]
Sep 10 16:02:53.493259 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.493410 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.493410 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6rk\" (UniqueName: \"kubernetes.io/projected/8e663046-1358-4eee-b727-957a72add662-kube-api-access-hm6rk\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.493410 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493406    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.493600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e663046-1358-4eee-b727-957a72add662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.493600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.493494    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.533566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.533533    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cb5f9f868-q8lx9"]
Sep 10 16:02:53.533766 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:02:53.533745    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9" podUID="70e1937b-6dec-4002-9ccf-d1f7ea622f6a"
Sep 10 16:02:53.569810 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.569773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:53.587133 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.587102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:53.594221 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6rk\" (UniqueName: \"kubernetes.io/projected/8e663046-1358-4eee-b727-957a72add662-kube-api-access-hm6rk\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.594381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.594468 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e663046-1358-4eee-b727-957a72add662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.594468 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.594836 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.594959 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.594874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.595168 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.595145    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e663046-1358-4eee-b727-957a72add662-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.597422 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.597401    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-ca\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.597507 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.597436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.597621 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.597602    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.597671 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.597631    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8e663046-1358-4eee-b727-957a72add662-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.629992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.629960    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6rk\" (UniqueName: \"kubernetes.io/projected/8e663046-1358-4eee-b727-957a72add662-kube-api-access-hm6rk\") pod \"cluster-proxy-proxy-agent-78d7944c5b-wfxcr\" (UID: \"8e663046-1358-4eee-b727-957a72add662\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.695211 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695183    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695315 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695242    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695315 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695293    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695319    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695365    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695399    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695429    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwpzt\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt\") pod \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\" (UID: \"70e1937b-6dec-4002-9ccf-d1f7ea622f6a\") "
Sep 10 16:02:53.695802 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695773    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:02:53.695957 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.695926    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:02:53.696087 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.696063    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:02:53.697889 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.697866    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt" (OuterVolumeSpecName: "kube-api-access-zwpzt") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "kube-api-access-zwpzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:02:53.697997 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.697946    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:02:53.698092 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.698066    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:02:53.698158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.698068    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "70e1937b-6dec-4002-9ccf-d1f7ea622f6a" (UID: "70e1937b-6dec-4002-9ccf-d1f7ea622f6a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:02:53.763224 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.763198    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796420    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-trusted-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796461    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-certificates\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796478    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-installation-pull-secrets\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796495    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-ca-trust-extracted\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796509    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwpzt\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-kube-api-access-zwpzt\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796523    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-bound-sa-token\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:53.796563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:53.796537    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-image-registry-private-configuration\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:54.096466 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.096434    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr"]
Sep 10 16:02:54.100235 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:54.100211    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e663046_1358_4eee_b727_957a72add662.slice/crio-77b0aa9de29c0e1c272a8d2899f4fe13f2dce231f138fb1b5b7e3ecdf5d0bad0 WatchSource:0}: Error finding container 77b0aa9de29c0e1c272a8d2899f4fe13f2dce231f138fb1b5b7e3ecdf5d0bad0: Status 404 returned error can't find the container with id 77b0aa9de29c0e1c272a8d2899f4fe13f2dce231f138fb1b5b7e3ecdf5d0bad0
Sep 10 16:02:54.575130 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.575095    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr" event={"ID":"8e663046-1358-4eee-b727-957a72add662","Type":"ContainerStarted","Data":"77b0aa9de29c0e1c272a8d2899f4fe13f2dce231f138fb1b5b7e3ecdf5d0bad0"}
Sep 10 16:02:54.577189 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.577154    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lns56" event={"ID":"d1e89918-a035-4d0c-9100-2db3b17e4310","Type":"ContainerStarted","Data":"497b6458ecab256cac1db317bbd75235d18dffe973bc80f3df57d81a7b3c602a"}
Sep 10 16:02:54.577356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.577206    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lns56" event={"ID":"d1e89918-a035-4d0c-9100-2db3b17e4310","Type":"ContainerStarted","Data":"df30cf4ebb1c3413866abe0b5313579fc357f6a362107285bd593f1cc5ba2936"}
Sep 10 16:02:54.579277 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.579247    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb5f9f868-q8lx9"
Sep 10 16:02:54.579396 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.579292    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z4t22" event={"ID":"4b3b5327-2865-4fdf-9b5a-be0f85204661","Type":"ContainerStarted","Data":"0266c4e93b519adf0d03132b5e46f55ead06905042211c627e821ef40dba5c12"}
Sep 10 16:02:54.579396 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.579354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z4t22" event={"ID":"4b3b5327-2865-4fdf-9b5a-be0f85204661","Type":"ContainerStarted","Data":"daedc3b055c0471f812706c4b689fd56c318d891110dd706d99a0f9e8099e4ce"}
Sep 10 16:02:54.579569 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.579543    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z4t22"
Sep 10 16:02:54.622704 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.622659    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lns56" podStartSLOduration=66.388060622 podStartE2EDuration="1m8.622641841s" totalImagesPullingTime="2.234581219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:54.621410307 +0000 UTC m=+69.854358409" watchObservedRunningTime="2026-09-10 16:02:54.622641841 +0000 UTC m=+69.855589945"
Sep 10 16:02:54.663843 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.663792    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z4t22" podStartSLOduration=33.493827683 podStartE2EDuration="35.663760181s" totalImagesPullingTime="2.169932498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:54.663581948 +0000 UTC m=+69.896530052" watchObservedRunningTime="2026-09-10 16:02:54.663760181 +0000 UTC m=+69.896708282"
Sep 10 16:02:54.729319 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.729285    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cb5f9f868-q8lx9"]
Sep 10 16:02:54.760187 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.760148    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cb5f9f868-q8lx9"]
Sep 10 16:02:54.803540 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:54.803501    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/70e1937b-6dec-4002-9ccf-d1f7ea622f6a-registry-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:02:55.206323 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.206285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:55.208712 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.208688    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d3ddef0-fd8c-4346-8b70-7cad9d7362bb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k2tkb\" (UID: \"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb\") " pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:55.265872 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.265833    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e1937b-6dec-4002-9ccf-d1f7ea622f6a" path="/var/lib/kubelet/pods/70e1937b-6dec-4002-9ccf-d1f7ea622f6a/volumes"
Sep 10 16:02:55.419385 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.419357    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6s77g\""
Sep 10 16:02:55.426707 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.426683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k2tkb"
Sep 10 16:02:55.992872 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:55.992849    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k2tkb"]
Sep 10 16:02:56.052155 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:02:56.052111    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3ddef0_fd8c_4346_8b70_7cad9d7362bb.slice/crio-f217d8e92fb45dd2723cdd3be6f8f53ea97d4c7d0c7b6bb7bd9135bb16e0a631 WatchSource:0}: Error finding container f217d8e92fb45dd2723cdd3be6f8f53ea97d4c7d0c7b6bb7bd9135bb16e0a631: Status 404 returned error can't find the container with id f217d8e92fb45dd2723cdd3be6f8f53ea97d4c7d0c7b6bb7bd9135bb16e0a631
Sep 10 16:02:56.594761 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:56.594520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c7krj" event={"ID":"e130d30f-5fe3-4bcd-825b-c52b6b9a6e27","Type":"ContainerStarted","Data":"013c52a7acc7f8329b4387f37937d6902ed28948e3b7f4e56aad3896219a0014"}
Sep 10 16:02:56.596426 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:56.596311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2tkb" event={"ID":"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb","Type":"ContainerStarted","Data":"baf3b9987d440c605c594c8f5528bbb7a638f92a1a9f0ffbe37ca2f39a3b443e"}
Sep 10 16:02:56.596426 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:56.596388    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2tkb" event={"ID":"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb","Type":"ContainerStarted","Data":"f217d8e92fb45dd2723cdd3be6f8f53ea97d4c7d0c7b6bb7bd9135bb16e0a631"}
Sep 10 16:02:56.635190 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:56.635137    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c7krj" podStartSLOduration=66.219925887 podStartE2EDuration="1m10.63511846s" totalImagesPullingTime="4.415192573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:56.634747039 +0000 UTC m=+71.867695143" watchObservedRunningTime="2026-09-10 16:02:56.63511846 +0000 UTC m=+71.868066559"
Sep 10 16:02:58.608175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:58.608130    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2tkb" event={"ID":"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb","Type":"ContainerStarted","Data":"96dfa10ec41610d8ffc06b04a8f8ebec2caf94b0eeee42d8e648b6bcf3ac4024"}
Sep 10 16:02:59.617347 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:59.617295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k2tkb" event={"ID":"8d3ddef0-fd8c-4346-8b70-7cad9d7362bb","Type":"ContainerStarted","Data":"38c595603d696133c481e06e2553635bf144ff45c4ab1e74aa7af5cfd96e2a12"}
Sep 10 16:02:59.618791 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:59.618768    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr" event={"ID":"8e663046-1358-4eee-b727-957a72add662","Type":"ContainerStarted","Data":"77b51f036a2ead8b1e831cb67456be237e2b627d05b3bbc90eada6cd831aee44"}
Sep 10 16:02:59.662962 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:02:59.662913    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k2tkb" podStartSLOduration=33.369156998 podStartE2EDuration="36.66289287s" totalImagesPullingTime="3.293735872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:02:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:02:59.661522376 +0000 UTC m=+74.894470481" watchObservedRunningTime="2026-09-10 16:02:59.66289287 +0000 UTC m=+74.895840973"
Sep 10 16:03:00.624822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:00.624780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr" event={"ID":"8e663046-1358-4eee-b727-957a72add662","Type":"ContainerStarted","Data":"0ea5f1758a002f2d15849a9818b79fed3911800d84cd364eae822119ab4e1ccd"}
Sep 10 16:03:00.625293 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:00.624835    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr" event={"ID":"8e663046-1358-4eee-b727-957a72add662","Type":"ContainerStarted","Data":"6ab9562cb60f4296ff44c04c0c26a3841b7dcdf856475caa29bc05acfe08bf1c"}
Sep 10 16:03:00.654716 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:00.654661    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78d7944c5b-wfxcr" podStartSLOduration=2.233340407 podStartE2EDuration="7.654647403s" totalImagesPullingTime="5.421306996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:03:00.653364742 +0000 UTC m=+75.886312848" watchObservedRunningTime="2026-09-10 16:03:00.654647403 +0000 UTC m=+75.887595499"
Sep 10 16:03:04.586371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.586315    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z4t22"
Sep 10 16:03:04.915865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.915787    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j2pgl"]
Sep 10 16:03:04.919270 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.919245    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.922950 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.922930    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 16:03:04.923066 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.922928    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 16:03:04.924204 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.924182    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 16:03:04.924312 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.924204    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 16:03:04.924312 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.924225    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 16:03:04.924312 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.924247    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 16:03:04.924312 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.924184    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-75jhs\""
Sep 10 16:03:04.965791 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-sys\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.965911 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-accelerators-collector-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.965911 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-root\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.965911 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-metrics-client-ca\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.966038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-textfile\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.966038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.966038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-wtmp\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.966038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.965994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:04.966038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:04.966010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzks2\" (UniqueName: \"kubernetes.io/projected/75e633de-eb82-4864-9a7e-ecabc4b64bce-kube-api-access-pzks2\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067243 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-textfile\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067243 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-wtmp\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzks2\" (UniqueName: \"kubernetes.io/projected/75e633de-eb82-4864-9a7e-ecabc4b64bce-kube-api-access-pzks2\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-sys\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-accelerators-collector-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-root\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-metrics-client-ca\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:03:05.067411    2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067412    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-wtmp\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:03:05.067597    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls podName:75e633de-eb82-4864-9a7e-ecabc4b64bce nodeName:}" failed. No retries permitted until 2026-09-10 16:03:05.567576202 +0000 UTC m=+80.800524301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls") pod "node-exporter-j2pgl" (UID: "75e633de-eb82-4864-9a7e-ecabc4b64bce") : secret "node-exporter-tls" not found
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067603    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-textfile\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067612    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-sys\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.067801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.067654    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/75e633de-eb82-4864-9a7e-ecabc4b64bce-root\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.068109 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.068088    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-accelerators-collector-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.068144 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.068088    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/75e633de-eb82-4864-9a7e-ecabc4b64bce-metrics-client-ca\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.069619 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.069598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.089393 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.089366    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzks2\" (UniqueName: \"kubernetes.io/projected/75e633de-eb82-4864-9a7e-ecabc4b64bce-kube-api-access-pzks2\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.571697 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.571662    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.573890 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.573869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/75e633de-eb82-4864-9a7e-ecabc4b64bce-node-exporter-tls\") pod \"node-exporter-j2pgl\" (UID: \"75e633de-eb82-4864-9a7e-ecabc4b64bce\") " pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.827846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:05.827754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j2pgl"
Sep 10 16:03:05.837497 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:03:05.837464    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e633de_eb82_4864_9a7e_ecabc4b64bce.slice/crio-0734e591aab007b544963318a772ee0313d090c1c495252d314e2bae807214c6 WatchSource:0}: Error finding container 0734e591aab007b544963318a772ee0313d090c1c495252d314e2bae807214c6: Status 404 returned error can't find the container with id 0734e591aab007b544963318a772ee0313d090c1c495252d314e2bae807214c6
Sep 10 16:03:06.645740 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:06.645711    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j2pgl" event={"ID":"75e633de-eb82-4864-9a7e-ecabc4b64bce","Type":"ContainerStarted","Data":"0734e591aab007b544963318a772ee0313d090c1c495252d314e2bae807214c6"}
Sep 10 16:03:07.650246 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:07.650137    2574 generic.go:356] "Generic (PLEG): container finished" podID="75e633de-eb82-4864-9a7e-ecabc4b64bce" containerID="8ee0199361c8448ef440845cfddbf4e2d712f62c049fccc548dd06fe0c56b7bd" exitCode=0
Sep 10 16:03:07.650246 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:07.650210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j2pgl" event={"ID":"75e633de-eb82-4864-9a7e-ecabc4b64bce","Type":"ContainerDied","Data":"8ee0199361c8448ef440845cfddbf4e2d712f62c049fccc548dd06fe0c56b7bd"}
Sep 10 16:03:08.655378 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:08.655322    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j2pgl" event={"ID":"75e633de-eb82-4864-9a7e-ecabc4b64bce","Type":"ContainerStarted","Data":"61222937b413029a7a6759559e80019d4e1df9e32f5b89894596965b17d436f1"}
Sep 10 16:03:08.655767 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:08.655424    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j2pgl" event={"ID":"75e633de-eb82-4864-9a7e-ecabc4b64bce","Type":"ContainerStarted","Data":"baecea38ae109eb8bb72119b41906474e054333a2f25ac77a48056f7e7deee3a"}
Sep 10 16:03:08.692081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:08.692031    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j2pgl" podStartSLOduration=3.8266185090000002 podStartE2EDuration="4.692016485s" totalImagesPullingTime="803.549603ms" totalInitContainerRuntime="61.848373ms" isStatefulPod=true podCreationTimestamp="2026-09-10 16:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:03:08.690131141 +0000 UTC m=+83.923079244" watchObservedRunningTime="2026-09-10 16:03:08.692016485 +0000 UTC m=+83.924964621"
Sep 10 16:03:09.152191 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.152162    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"]
Sep 10 16:03:09.154269 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.154248    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.156879 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.156855    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 16:03:09.157018 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.156998    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nms7p\""
Sep 10 16:03:09.168373 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.168351    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"]
Sep 10 16:03:09.201132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.201106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9ln\" (UniqueName: \"kubernetes.io/projected/29496c12-2be0-411a-a331-ac408d8ebfd5-kube-api-access-ff9ln\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.201228 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.201134    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.301778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.301752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9ln\" (UniqueName: \"kubernetes.io/projected/29496c12-2be0-411a-a331-ac408d8ebfd5-kube-api-access-ff9ln\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.301778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.301780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.301919 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:03:09.301883    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 16:03:09.301954 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:03:09.301932    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert podName:29496c12-2be0-411a-a331-ac408d8ebfd5 nodeName:}" failed. No retries permitted until 2026-09-10 16:03:09.801917119 +0000 UTC m=+85.034865199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-x7fn9" (UID: "29496c12-2be0-411a-a331-ac408d8ebfd5") : secret "monitoring-plugin-cert" not found
Sep 10 16:03:09.317905 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.317878    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9ln\" (UniqueName: \"kubernetes.io/projected/29496c12-2be0-411a-a331-ac408d8ebfd5-kube-api-access-ff9ln\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.568553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.568525    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"]
Sep 10 16:03:09.570759 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.570741    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.577530 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.577500    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 16:03:09.577687 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.577643    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 16:03:09.577791 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.577776    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b6ncdhfrcjlmu\""
Sep 10 16:03:09.579085 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.579069    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 16:03:09.579288 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.579276    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pm5zn\""
Sep 10 16:03:09.579935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.579920    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 16:03:09.603608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603717 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-tls\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603717 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603695    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhzx\" (UniqueName: \"kubernetes.io/projected/842fad69-61de-4835-b3ba-3837faba2c9e-kube-api-access-6xhzx\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603799 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-metrics-server-audit-profiles\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603799 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/842fad69-61de-4835-b3ba-3837faba2c9e-audit-log\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603799 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603788    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-client-certs\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.603903 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.603810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-client-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.604950 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.604931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"]
Sep 10 16:03:09.704965 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.704933    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-client-certs\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.704981    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-client-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-tls\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhzx\" (UniqueName: \"kubernetes.io/projected/842fad69-61de-4835-b3ba-3837faba2c9e-kube-api-access-6xhzx\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-metrics-server-audit-profiles\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705411 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/842fad69-61de-4835-b3ba-3837faba2c9e-audit-log\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705797 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705773    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/842fad69-61de-4835-b3ba-3837faba2c9e-audit-log\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.705868 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.705820    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.706099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.706079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/842fad69-61de-4835-b3ba-3837faba2c9e-metrics-server-audit-profiles\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.707523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.707504    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-client-certs\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.707920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.707898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-secret-metrics-server-tls\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.708010 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.707974    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/842fad69-61de-4835-b3ba-3837faba2c9e-client-ca-bundle\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.727862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.727837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhzx\" (UniqueName: \"kubernetes.io/projected/842fad69-61de-4835-b3ba-3837faba2c9e-kube-api-access-6xhzx\") pod \"metrics-server-bb8cc74d9-ndlnz\" (UID: \"842fad69-61de-4835-b3ba-3837faba2c9e\") " pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:09.805814 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.805784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.807938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.807915    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/29496c12-2be0-411a-a331-ac408d8ebfd5-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-x7fn9\" (UID: \"29496c12-2be0-411a-a331-ac408d8ebfd5\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:09.880041 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:09.879973    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:10.063759 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:10.063727    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:10.195571 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:10.195499    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"]
Sep 10 16:03:10.199231 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:03:10.199206    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod842fad69_61de_4835_b3ba_3837faba2c9e.slice/crio-edd4221f5614814ed1cb122c891d02a7a7df090369a247cada6230ddedaf1994 WatchSource:0}: Error finding container edd4221f5614814ed1cb122c891d02a7a7df090369a247cada6230ddedaf1994: Status 404 returned error can't find the container with id edd4221f5614814ed1cb122c891d02a7a7df090369a247cada6230ddedaf1994
Sep 10 16:03:10.352477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:10.352450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"]
Sep 10 16:03:10.353647 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:03:10.353620    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29496c12_2be0_411a_a331_ac408d8ebfd5.slice/crio-082bcb447b7131822cbd6dd82c5a06d42eb8268e16d5ba7b74a70d60a6275058 WatchSource:0}: Error finding container 082bcb447b7131822cbd6dd82c5a06d42eb8268e16d5ba7b74a70d60a6275058: Status 404 returned error can't find the container with id 082bcb447b7131822cbd6dd82c5a06d42eb8268e16d5ba7b74a70d60a6275058
Sep 10 16:03:10.662971 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:10.662936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9" event={"ID":"29496c12-2be0-411a-a331-ac408d8ebfd5","Type":"ContainerStarted","Data":"082bcb447b7131822cbd6dd82c5a06d42eb8268e16d5ba7b74a70d60a6275058"}
Sep 10 16:03:10.664039 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:10.664019    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz" event={"ID":"842fad69-61de-4835-b3ba-3837faba2c9e","Type":"ContainerStarted","Data":"edd4221f5614814ed1cb122c891d02a7a7df090369a247cada6230ddedaf1994"}
Sep 10 16:03:12.672369 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.672314    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz" event={"ID":"842fad69-61de-4835-b3ba-3837faba2c9e","Type":"ContainerStarted","Data":"bae1b50fdccc9d0b329540cc6263725cad972e32816c2023c94dc1e77c756bf3"}
Sep 10 16:03:12.673888 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.673865    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9" event={"ID":"29496c12-2be0-411a-a331-ac408d8ebfd5","Type":"ContainerStarted","Data":"845b98943bee012a11973a3303a670ecf297d6bccda400f7b18fdd39e1450a94"}
Sep 10 16:03:12.674096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.674068    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:12.678683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.678665    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9"
Sep 10 16:03:12.701046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.701006    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz" podStartSLOduration=1.952158388 podStartE2EDuration="3.700992921s" totalImagesPullingTime="1.748834533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:03:12.699551929 +0000 UTC m=+87.932500031" watchObservedRunningTime="2026-09-10 16:03:12.700992921 +0000 UTC m=+87.933941022"
Sep 10 16:03:12.718694 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:12.718656    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-x7fn9" podStartSLOduration=2.122901007 podStartE2EDuration="3.71864302s" totalImagesPullingTime="1.595742013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:03:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:03:12.716715544 +0000 UTC m=+87.949663648" watchObservedRunningTime="2026-09-10 16:03:12.71864302 +0000 UTC m=+87.951591121"
Sep 10 16:03:23.572781 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:23.572704    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c67tz"
Sep 10 16:03:29.880118 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:29.880078    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:29.880523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:29.880143    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:31.741269 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:31.741194    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b8f7029-b8bd-469a-8019-72c207ff1b9a" containerID="dad9f20ea889ce541c82e3f14ef26d4e48a70eecde433eda03243847ec5a752e" exitCode=0
Sep 10 16:03:31.741610 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:31.741265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx" event={"ID":"9b8f7029-b8bd-469a-8019-72c207ff1b9a","Type":"ContainerDied","Data":"dad9f20ea889ce541c82e3f14ef26d4e48a70eecde433eda03243847ec5a752e"}
Sep 10 16:03:31.741610 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:31.741571    2574 scope.go:117] "RemoveContainer" containerID="dad9f20ea889ce541c82e3f14ef26d4e48a70eecde433eda03243847ec5a752e"
Sep 10 16:03:32.746142 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:32.746107    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-n22kx" event={"ID":"9b8f7029-b8bd-469a-8019-72c207ff1b9a","Type":"ContainerStarted","Data":"b1aa0948f8d2f967b1894d69dbc09658638420dbe55c65b2e8486141f80ed648"}
Sep 10 16:03:40.775589 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:40.775560    2574 generic.go:356] "Generic (PLEG): container finished" podID="1934703d-0bf7-4a49-a6dd-9e79bf4d0806" containerID="b9b74b85ef5558932f6c5deb02cd0c8ab0f5c74c941c99e6fbb6c1e44a603f28" exitCode=0
Sep 10 16:03:40.775929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:40.775632    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd" event={"ID":"1934703d-0bf7-4a49-a6dd-9e79bf4d0806","Type":"ContainerDied","Data":"b9b74b85ef5558932f6c5deb02cd0c8ab0f5c74c941c99e6fbb6c1e44a603f28"}
Sep 10 16:03:40.775929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:40.775918    2574 scope.go:117] "RemoveContainer" containerID="b9b74b85ef5558932f6c5deb02cd0c8ab0f5c74c941c99e6fbb6c1e44a603f28"
Sep 10 16:03:41.780516 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:41.780478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-f8jqd" event={"ID":"1934703d-0bf7-4a49-a6dd-9e79bf4d0806","Type":"ContainerStarted","Data":"439f4cd1c84405b9d223ae318321422a81d71baf55f0df19dd48f0717efc25df"}
Sep 10 16:03:49.886089 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:49.886058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:03:49.889874 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:03:49.889852    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bb8cc74d9-ndlnz"
Sep 10 16:06:45.170836 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:06:45.170803    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:06:45.177201 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:06:45.177180    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:06:45.178197 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:06:45.178169    2574 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 16:11:22.195021 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.194988    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ll92v"]
Sep 10 16:11:22.196821 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.196804    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.199849 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.199824    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 16:11:22.201129 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.201111    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 16:11:22.201220 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.201137    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4jr9x\""
Sep 10 16:11:22.211602 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.211580    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ll92v"]
Sep 10 16:11:22.305144 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.305108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbd7t\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-kube-api-access-mbd7t\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.305283 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.305179    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-bound-sa-token\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.405644 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.405616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbd7t\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-kube-api-access-mbd7t\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.405809 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.405687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-bound-sa-token\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.416482 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.416451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbd7t\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-kube-api-access-mbd7t\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.416709 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.416692    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3c914a-1b6f-409b-8255-3844ff566289-bound-sa-token\") pod \"cert-manager-55b4d954d5-ll92v\" (UID: \"0a3c914a-1b6f-409b-8255-3844ff566289\") " pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.505337 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.505262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ll92v"
Sep 10 16:11:22.780900 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.780872    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ll92v"]
Sep 10 16:11:22.784777 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:11:22.784747    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a3c914a_1b6f_409b_8255_3844ff566289.slice/crio-1af0ea3a5db89f7912edf9dd9340ede59738b8ebc6d30335ac6bfae91dd8e00c WatchSource:0}: Error finding container 1af0ea3a5db89f7912edf9dd9340ede59738b8ebc6d30335ac6bfae91dd8e00c: Status 404 returned error can't find the container with id 1af0ea3a5db89f7912edf9dd9340ede59738b8ebc6d30335ac6bfae91dd8e00c
Sep 10 16:11:22.786698 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:22.786678    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:11:23.340804 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:23.340767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ll92v" event={"ID":"0a3c914a-1b6f-409b-8255-3844ff566289","Type":"ContainerStarted","Data":"1af0ea3a5db89f7912edf9dd9340ede59738b8ebc6d30335ac6bfae91dd8e00c"}
Sep 10 16:11:25.473925 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.473895    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"]
Sep 10 16:11:25.476158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.476136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.479007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.478986    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:11:25.480314 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.480292    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zm5wd\""
Sep 10 16:11:25.480418 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.480295    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 16:11:25.490873 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.490851    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"]
Sep 10 16:11:25.530177 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.530147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.530299 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.530198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47hsn\" (UniqueName: \"kubernetes.io/projected/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-kube-api-access-47hsn\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.631576 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.631536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.631576 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.631588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47hsn\" (UniqueName: \"kubernetes.io/projected/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-kube-api-access-47hsn\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.631993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.631967    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.648370 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.648323    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47hsn\" (UniqueName: \"kubernetes.io/projected/58fa5d15-ad01-4864-8abb-d7d8afef4ea4-kube-api-access-47hsn\") pod \"openshift-lws-operator-fd8ccff4c-tjmx4\" (UID: \"58fa5d15-ad01-4864-8abb-d7d8afef4ea4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:25.786985 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:25.786962    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"
Sep 10 16:11:26.058096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:26.058066    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4"]
Sep 10 16:11:26.060487 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:11:26.060457    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58fa5d15_ad01_4864_8abb_d7d8afef4ea4.slice/crio-55539024e630515ef3d9fb15b1d15707679a91aa06c4fde74a02b0f49aeb4de1 WatchSource:0}: Error finding container 55539024e630515ef3d9fb15b1d15707679a91aa06c4fde74a02b0f49aeb4de1: Status 404 returned error can't find the container with id 55539024e630515ef3d9fb15b1d15707679a91aa06c4fde74a02b0f49aeb4de1
Sep 10 16:11:26.352427 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:26.352349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4" event={"ID":"58fa5d15-ad01-4864-8abb-d7d8afef4ea4","Type":"ContainerStarted","Data":"55539024e630515ef3d9fb15b1d15707679a91aa06c4fde74a02b0f49aeb4de1"}
Sep 10 16:11:26.353778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:26.353752    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ll92v" event={"ID":"0a3c914a-1b6f-409b-8255-3844ff566289","Type":"ContainerStarted","Data":"e1d0f5bb267d264e96fa5c9939eef685adb0d01a07e8ee4b7e86b94b88cf8ece"}
Sep 10 16:11:26.373816 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:26.373774    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ll92v" podStartSLOduration=1.398714589 podStartE2EDuration="4.373759546s" totalImagesPullingTime="2.975044957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:11:26.371974667 +0000 UTC m=+581.604922768" watchObservedRunningTime="2026-09-10 16:11:26.373759546 +0000 UTC m=+581.606707647"
Sep 10 16:11:29.366702 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:29.366671    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4" event={"ID":"58fa5d15-ad01-4864-8abb-d7d8afef4ea4","Type":"ContainerStarted","Data":"0d36b67746f61ff294c86bdb4da926e8dd910fa1edd13d8a55656f48f533200f"}
Sep 10 16:11:29.386182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:29.386135    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tjmx4" podStartSLOduration=2.079212535 podStartE2EDuration="4.386119821s" totalImagesPullingTime="2.306907286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:11:29.385816822 +0000 UTC m=+584.618764925" watchObservedRunningTime="2026-09-10 16:11:29.386119821 +0000 UTC m=+584.619067924"
Sep 10 16:11:45.196688 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:45.196658    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:11:45.201485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:45.201461    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:11:48.265933 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.265899    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"]
Sep 10 16:11:48.274311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.274291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.277909 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.277886    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 16:11:48.278034 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.277950    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 16:11:48.278152 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.278137    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9p2dd\""
Sep 10 16:11:48.278439 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.278414    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 16:11:48.278566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.278517    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 16:11:48.353379 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.353350    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"]
Sep 10 16:11:48.385410 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.385384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22gw\" (UniqueName: \"kubernetes.io/projected/f5a24fcc-3722-4012-a038-5446012342cc-kube-api-access-k22gw\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.385548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.385425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-manifests\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.385548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.385454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-tmp\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.385548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.385510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.385548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.385527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.486843 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.486817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.486964 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.486847    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.486964 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.486879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22gw\" (UniqueName: \"kubernetes.io/projected/f5a24fcc-3722-4012-a038-5446012342cc-kube-api-access-k22gw\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.487175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.487157    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-manifests\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.487229 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.487212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-tmp\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.487538 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.487514    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-manifests\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.489258 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.489236    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5a24fcc-3722-4012-a038-5446012342cc-tmp\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.489475 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.489454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.489548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.489463    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f5a24fcc-3722-4012-a038-5446012342cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.534031 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.534003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22gw\" (UniqueName: \"kubernetes.io/projected/f5a24fcc-3722-4012-a038-5446012342cc-kube-api-access-k22gw\") pod \"opendatahub-operator-controller-manager-67779d4d4c-k4dn4\" (UID: \"f5a24fcc-3722-4012-a038-5446012342cc\") " pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.583738 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.583716    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:48.906221 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:48.906197    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"]
Sep 10 16:11:48.908594 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:11:48.908567    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5a24fcc_3722_4012_a038_5446012342cc.slice/crio-fac956c9d8a05dc740b87e9b2c7ea73025e8c294523b4ff158d95d67921ffa15 WatchSource:0}: Error finding container fac956c9d8a05dc740b87e9b2c7ea73025e8c294523b4ff158d95d67921ffa15: Status 404 returned error can't find the container with id fac956c9d8a05dc740b87e9b2c7ea73025e8c294523b4ff158d95d67921ffa15
Sep 10 16:11:49.438793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:49.438747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4" event={"ID":"f5a24fcc-3722-4012-a038-5446012342cc","Type":"ContainerStarted","Data":"fac956c9d8a05dc740b87e9b2c7ea73025e8c294523b4ff158d95d67921ffa15"}
Sep 10 16:11:52.453529 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:52.453496    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4" event={"ID":"f5a24fcc-3722-4012-a038-5446012342cc","Type":"ContainerStarted","Data":"99d9a0179a9dd4df450e5f149fb7f9c940ddea29210feee65294de3f95b08523"}
Sep 10 16:11:53.458371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:53.458317    2574 generic.go:356] "Generic (PLEG): container finished" podID="f5a24fcc-3722-4012-a038-5446012342cc" containerID="99d9a0179a9dd4df450e5f149fb7f9c940ddea29210feee65294de3f95b08523" exitCode=0
Sep 10 16:11:53.458371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:53.458363    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4" event={"ID":"f5a24fcc-3722-4012-a038-5446012342cc","Type":"ContainerDied","Data":"99d9a0179a9dd4df450e5f149fb7f9c940ddea29210feee65294de3f95b08523"}
Sep 10 16:11:54.464730 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:54.464693    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4" event={"ID":"f5a24fcc-3722-4012-a038-5446012342cc","Type":"ContainerStarted","Data":"fbbb1a39288706c02a0c9d920ef71bc9c1bcd4217a9426a240a2bd8d69a43a37"}
Sep 10 16:11:54.465204 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:54.464824    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:11:54.501930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:11:54.501884    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4" podStartSLOduration=2.095383225 podStartE2EDuration="6.501870663s" totalImagesPullingTime="3.38406528s" totalInitContainerRuntime="1.022422158s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:11:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:11:54.501314986 +0000 UTC m=+609.734263089" watchObservedRunningTime="2026-09-10 16:11:54.501870663 +0000 UTC m=+609.734818765"
Sep 10 16:12:05.470273 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:05.470244    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67779d4d4c-k4dn4"
Sep 10 16:12:08.767827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.767743    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"]
Sep 10 16:12:08.776048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.776025    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:08.779250 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.779215    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 16:12:08.779628 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.779613    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9hk9d\""
Sep 10 16:12:08.780065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.780049    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 16:12:08.780115 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.780071    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 16:12:08.783791 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.783768    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 16:12:08.789719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.789695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"]
Sep 10 16:12:08.943527 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.943493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8c8x\" (UniqueName: \"kubernetes.io/projected/fbebe6d2-f166-44ab-8043-27172ba65816-kube-api-access-x8c8x\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:08.943705 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.943544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:08.943705 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:08.943604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbebe6d2-f166-44ab-8043-27172ba65816-tmp\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.044600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.044575    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8c8x\" (UniqueName: \"kubernetes.io/projected/fbebe6d2-f166-44ab-8043-27172ba65816-kube-api-access-x8c8x\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.044746 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.044628    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.044746 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.044654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbebe6d2-f166-44ab-8043-27172ba65816-tmp\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.044863 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:12:09.044750    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 16:12:09.044863 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:12:09.044827    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs podName:fbebe6d2-f166-44ab-8043-27172ba65816 nodeName:}" failed. No retries permitted until 2026-09-10 16:12:09.544805604 +0000 UTC m=+624.777753700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs") pod "kube-auth-proxy-74797cb87b-xmjxn" (UID: "fbebe6d2-f166-44ab-8043-27172ba65816") : secret "kube-auth-proxy-tls" not found
Sep 10 16:12:09.046902 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.046881    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbebe6d2-f166-44ab-8043-27172ba65816-tmp\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.064837 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.064810    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8c8x\" (UniqueName: \"kubernetes.io/projected/fbebe6d2-f166-44ab-8043-27172ba65816-kube-api-access-x8c8x\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.549126 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.549091    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.551486 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.551463    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbebe6d2-f166-44ab-8043-27172ba65816-tls-certs\") pod \"kube-auth-proxy-74797cb87b-xmjxn\" (UID: \"fbebe6d2-f166-44ab-8043-27172ba65816\") " pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.685111 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.685077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"
Sep 10 16:12:09.976938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:09.976865    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn"]
Sep 10 16:12:09.979575 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:12:09.979546    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbebe6d2_f166_44ab_8043_27172ba65816.slice/crio-8aff7751513956f9e734a4d48d5e13b594803664810222121e9893abe7a56df8 WatchSource:0}: Error finding container 8aff7751513956f9e734a4d48d5e13b594803664810222121e9893abe7a56df8: Status 404 returned error can't find the container with id 8aff7751513956f9e734a4d48d5e13b594803664810222121e9893abe7a56df8
Sep 10 16:12:10.527174 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:10.527137    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn" event={"ID":"fbebe6d2-f166-44ab-8043-27172ba65816","Type":"ContainerStarted","Data":"8aff7751513956f9e734a4d48d5e13b594803664810222121e9893abe7a56df8"}
Sep 10 16:12:13.542664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:13.542631    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn" event={"ID":"fbebe6d2-f166-44ab-8043-27172ba65816","Type":"ContainerStarted","Data":"d25e53a71b5f05b6e1511e9166c7b0b9b0053078fe1f743005f9e1603bb40b03"}
Sep 10 16:12:13.570058 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:13.570006    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74797cb87b-xmjxn" podStartSLOduration=2.409520724 podStartE2EDuration="5.56999102s" totalImagesPullingTime="3.160470296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:12:13.567895948 +0000 UTC m=+628.800844051" watchObservedRunningTime="2026-09-10 16:12:13.56999102 +0000 UTC m=+628.802939119"
Sep 10 16:12:17.186909 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.186871    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"]
Sep 10 16:12:17.190231 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.190215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.194447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.194425    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 16:12:17.195153 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.195132    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 16:12:17.195272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.195163    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nlsw9\""
Sep 10 16:12:17.195272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.195139    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 16:12:17.210866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.210843    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"]
Sep 10 16:12:17.305622 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.305597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-metrics-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.305756 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.305631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p7c4\" (UniqueName: \"kubernetes.io/projected/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-kube-api-access-5p7c4\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.305756 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.305688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-manager-config\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.305756 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.305714    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.406934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.406902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-manager-config\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.407120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.407092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.407238 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.407221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-metrics-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.407273 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.407258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p7c4\" (UniqueName: \"kubernetes.io/projected/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-kube-api-access-5p7c4\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.407586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.407560    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-manager-config\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.409514 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.409494    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-metrics-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.409630 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.409559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-cert\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.417350 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.417307    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p7c4\" (UniqueName: \"kubernetes.io/projected/a783d7f6-3895-4d0c-aea7-dc1b08af7a8d-kube-api-access-5p7c4\") pod \"lws-controller-manager-84479dc8dd-vhlpl\" (UID: \"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d\") " pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.499413 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.499348    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:17.773263 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:17.773238    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"]
Sep 10 16:12:17.775749 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:12:17.775720    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda783d7f6_3895_4d0c_aea7_dc1b08af7a8d.slice/crio-63e0ed01c84d84c1a68ec700e426d73f7371acafc7fb36d3a16f227a57170757 WatchSource:0}: Error finding container 63e0ed01c84d84c1a68ec700e426d73f7371acafc7fb36d3a16f227a57170757: Status 404 returned error can't find the container with id 63e0ed01c84d84c1a68ec700e426d73f7371acafc7fb36d3a16f227a57170757
Sep 10 16:12:18.564255 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:18.564220    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl" event={"ID":"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d","Type":"ContainerStarted","Data":"63e0ed01c84d84c1a68ec700e426d73f7371acafc7fb36d3a16f227a57170757"}
Sep 10 16:12:19.572139 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:19.572050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl" event={"ID":"a783d7f6-3895-4d0c-aea7-dc1b08af7a8d","Type":"ContainerStarted","Data":"527825f24dff39aff1219bee0c28c5e3909b4d975e1cb865071907c1d45d3b40"}
Sep 10 16:12:19.572668 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:19.572205    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:12:19.595601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:19.595550    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl" podStartSLOduration=1.064213605 podStartE2EDuration="2.595512387s" totalImagesPullingTime="1.531298782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:12:19.59459782 +0000 UTC m=+634.827545922" watchObservedRunningTime="2026-09-10 16:12:19.595512387 +0000 UTC m=+634.828460489"
Sep 10 16:12:30.579765 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:12:30.579728    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84479dc8dd-vhlpl"
Sep 10 16:13:48.544887 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.544809    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"]
Sep 10 16:13:48.546811 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.546795    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.550242 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.550220    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 16:13:48.550377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.550219    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 16:13:48.550377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.550219    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 16:13:48.551492 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.551472    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qlltw\""
Sep 10 16:13:48.551492 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.551484    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 16:13:48.567044 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.567021    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"]
Sep 10 16:13:48.602989 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.602960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.603107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.603006    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5494698d-d5b0-494d-8485-4cf9293657e6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.603107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.603087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qngd7\" (UniqueName: \"kubernetes.io/projected/5494698d-d5b0-494d-8485-4cf9293657e6-kube-api-access-qngd7\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.703508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.703482    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qngd7\" (UniqueName: \"kubernetes.io/projected/5494698d-d5b0-494d-8485-4cf9293657e6-kube-api-access-qngd7\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.703608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.703517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.703608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.703548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5494698d-d5b0-494d-8485-4cf9293657e6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.703679 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:13:48.703627    2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 16:13:48.703712 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:13:48.703694    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert podName:5494698d-d5b0-494d-8485-4cf9293657e6 nodeName:}" failed. No retries permitted until 2026-09-10 16:13:49.20367906 +0000 UTC m=+724.436627154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-mj82g" (UID: "5494698d-d5b0-494d-8485-4cf9293657e6") : secret "plugin-serving-cert" not found
Sep 10 16:13:48.704071 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.704056    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5494698d-d5b0-494d-8485-4cf9293657e6-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:48.716546 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:48.716519    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qngd7\" (UniqueName: \"kubernetes.io/projected/5494698d-d5b0-494d-8485-4cf9293657e6-kube-api-access-qngd7\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:49.208474 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:49.208442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:49.210767 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:49.210748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5494698d-d5b0-494d-8485-4cf9293657e6-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-mj82g\" (UID: \"5494698d-d5b0-494d-8485-4cf9293657e6\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:49.455444 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:49.455410    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"
Sep 10 16:13:49.730921 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:49.730731    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g"]
Sep 10 16:13:49.733385 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:13:49.733353    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5494698d_d5b0_494d_8485_4cf9293657e6.slice/crio-daf16ee3c8c547094e1216b116c4c81b374b8335fa09b28fae3f290983e6a56a WatchSource:0}: Error finding container daf16ee3c8c547094e1216b116c4c81b374b8335fa09b28fae3f290983e6a56a: Status 404 returned error can't find the container with id daf16ee3c8c547094e1216b116c4c81b374b8335fa09b28fae3f290983e6a56a
Sep 10 16:13:49.928595 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:49.928560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g" event={"ID":"5494698d-d5b0-494d-8485-4cf9293657e6","Type":"ContainerStarted","Data":"daf16ee3c8c547094e1216b116c4c81b374b8335fa09b28fae3f290983e6a56a"}
Sep 10 16:13:54.953768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:54.953729    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g" event={"ID":"5494698d-d5b0-494d-8485-4cf9293657e6","Type":"ContainerStarted","Data":"eab5e462bbe25bafda1c1cdc8ea984194394225f29f79d0e69c0f0b3f7b553b7"}
Sep 10 16:13:54.971537 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:13:54.971489    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-mj82g" podStartSLOduration=2.695944978 podStartE2EDuration="6.97147612s" totalImagesPullingTime="4.275531142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:13:54.970952685 +0000 UTC m=+730.203900788" watchObservedRunningTime="2026-09-10 16:13:54.97147612 +0000 UTC m=+730.204424240"
Sep 10 16:14:48.059415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.059381    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:48.062308 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.062284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:48.065353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.065311    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h5k2r\""
Sep 10 16:14:48.072161 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.072137    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:48.169693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.169661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94lfm\" (UniqueName: \"kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm\") pod \"authorino-7bcfd8f9cd-lvv2b\" (UID: \"6b453de6-def7-4559-900f-c84529a0838b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:48.270245 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.270212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94lfm\" (UniqueName: \"kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm\") pod \"authorino-7bcfd8f9cd-lvv2b\" (UID: \"6b453de6-def7-4559-900f-c84529a0838b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:48.280859 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.280835    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94lfm\" (UniqueName: \"kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm\") pod \"authorino-7bcfd8f9cd-lvv2b\" (UID: \"6b453de6-def7-4559-900f-c84529a0838b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:48.348231 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.348146    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:14:48.351215 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.351190    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:14:48.363703 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.363681    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:14:48.370470 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.370449    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:48.370668 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.370650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rg4k\" (UniqueName: \"kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k\") pod \"authorino-59c974bd48-7rpm6\" (UID: \"807ad561-4096-417e-aeda-90ede3c1aaca\") " pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:14:48.472019 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.471989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rg4k\" (UniqueName: \"kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k\") pod \"authorino-59c974bd48-7rpm6\" (UID: \"807ad561-4096-417e-aeda-90ede3c1aaca\") " pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:14:48.481526 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.481500    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rg4k\" (UniqueName: \"kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k\") pod \"authorino-59c974bd48-7rpm6\" (UID: \"807ad561-4096-417e-aeda-90ede3c1aaca\") " pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:14:48.643651 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.643627    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:48.645683 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:14:48.645655    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b453de6_def7_4559_900f_c84529a0838b.slice/crio-96ad37b6c115af222d3af83fe20c5c257b696325e1f0f0a7eafdf95c1469442a WatchSource:0}: Error finding container 96ad37b6c115af222d3af83fe20c5c257b696325e1f0f0a7eafdf95c1469442a: Status 404 returned error can't find the container with id 96ad37b6c115af222d3af83fe20c5c257b696325e1f0f0a7eafdf95c1469442a
Sep 10 16:14:48.660179 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.660159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:14:48.938112 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:48.938041    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:14:48.941554 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:14:48.941520    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod807ad561_4096_417e_aeda_90ede3c1aaca.slice/crio-34f7149a5bf9b5b296e56a7d2a229542481212be328c436b30858903172fdcab WatchSource:0}: Error finding container 34f7149a5bf9b5b296e56a7d2a229542481212be328c436b30858903172fdcab: Status 404 returned error can't find the container with id 34f7149a5bf9b5b296e56a7d2a229542481212be328c436b30858903172fdcab
Sep 10 16:14:49.176171 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:49.176134    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" event={"ID":"6b453de6-def7-4559-900f-c84529a0838b","Type":"ContainerStarted","Data":"96ad37b6c115af222d3af83fe20c5c257b696325e1f0f0a7eafdf95c1469442a"}
Sep 10 16:14:49.177298 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:49.177273    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-7rpm6" event={"ID":"807ad561-4096-417e-aeda-90ede3c1aaca","Type":"ContainerStarted","Data":"34f7149a5bf9b5b296e56a7d2a229542481212be328c436b30858903172fdcab"}
Sep 10 16:14:52.192638 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:52.192600    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" event={"ID":"6b453de6-def7-4559-900f-c84529a0838b","Type":"ContainerStarted","Data":"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"}
Sep 10 16:14:52.194202 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:52.194168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-7rpm6" event={"ID":"807ad561-4096-417e-aeda-90ede3c1aaca","Type":"ContainerStarted","Data":"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"}
Sep 10 16:14:52.211184 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:52.211142    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" podStartSLOduration=1.403325027 podStartE2EDuration="4.211129045s" totalImagesPullingTime="2.807804018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:14:52.210433169 +0000 UTC m=+787.443381271" watchObservedRunningTime="2026-09-10 16:14:52.211129045 +0000 UTC m=+787.444077148"
Sep 10 16:14:52.230501 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:52.230449    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-7rpm6" podStartSLOduration=1.707434533 podStartE2EDuration="4.230434281s" totalImagesPullingTime="2.522999748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:14:52.230432205 +0000 UTC m=+787.463380308" watchObservedRunningTime="2026-09-10 16:14:52.230434281 +0000 UTC m=+787.463382383"
Sep 10 16:14:52.258828 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:52.258797    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:54.202432 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:54.202391    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" podUID="6b453de6-def7-4559-900f-c84529a0838b" containerName="authorino" containerID="cri-o://ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea" gracePeriod=30
Sep 10 16:14:54.435586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:54.435561    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:54.521891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:54.521811    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94lfm\" (UniqueName: \"kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm\") pod \"6b453de6-def7-4559-900f-c84529a0838b\" (UID: \"6b453de6-def7-4559-900f-c84529a0838b\") "
Sep 10 16:14:54.523940 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:54.523910    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm" (OuterVolumeSpecName: "kube-api-access-94lfm") pod "6b453de6-def7-4559-900f-c84529a0838b" (UID: "6b453de6-def7-4559-900f-c84529a0838b"). InnerVolumeSpecName "kube-api-access-94lfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:14:54.623079 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:54.623035    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94lfm\" (UniqueName: \"kubernetes.io/projected/6b453de6-def7-4559-900f-c84529a0838b-kube-api-access-94lfm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:14:55.207464 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.207431    2574 generic.go:356] "Generic (PLEG): container finished" podID="6b453de6-def7-4559-900f-c84529a0838b" containerID="ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea" exitCode=0
Sep 10 16:14:55.207866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.207477    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b"
Sep 10 16:14:55.207866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.207483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" event={"ID":"6b453de6-def7-4559-900f-c84529a0838b","Type":"ContainerDied","Data":"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"}
Sep 10 16:14:55.207866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.207512    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-lvv2b" event={"ID":"6b453de6-def7-4559-900f-c84529a0838b","Type":"ContainerDied","Data":"96ad37b6c115af222d3af83fe20c5c257b696325e1f0f0a7eafdf95c1469442a"}
Sep 10 16:14:55.207866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.207528    2574 scope.go:117] "RemoveContainer" containerID="ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"
Sep 10 16:14:55.215794 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.215775    2574 scope.go:117] "RemoveContainer" containerID="ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"
Sep 10 16:14:55.216058 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:14:55.216039    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea\": container with ID starting with ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea not found: ID does not exist" containerID="ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"
Sep 10 16:14:55.216104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.216069    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea"} err="failed to get container status \"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea\": rpc error: code = NotFound desc = could not find container \"ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea\": container with ID starting with ec083851ccd17bc50db92ae679db18847528af6f593aada0176615274e1ce5ea not found: ID does not exist"
Sep 10 16:14:55.229824 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.229801    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:55.236844 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.236826    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-lvv2b"]
Sep 10 16:14:55.265561 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:14:55.265538    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b453de6-def7-4559-900f-c84529a0838b" path="/var/lib/kubelet/pods/6b453de6-def7-4559-900f-c84529a0838b/volumes"
Sep 10 16:15:15.980283 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.980194    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:15.980851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.980612    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b453de6-def7-4559-900f-c84529a0838b" containerName="authorino"
Sep 10 16:15:15.980851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.980631    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b453de6-def7-4559-900f-c84529a0838b" containerName="authorino"
Sep 10 16:15:15.980851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.980720    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b453de6-def7-4559-900f-c84529a0838b" containerName="authorino"
Sep 10 16:15:15.983589 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.983568    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:15.990699 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:15.990673    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:16.087869 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.087838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vccv2\" (UniqueName: \"kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2\") pod \"authorino-557f876f69-q57tr\" (UID: \"608d6715-ff12-41df-a9e7-58918a473190\") " pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:16.188839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.188806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vccv2\" (UniqueName: \"kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2\") pod \"authorino-557f876f69-q57tr\" (UID: \"608d6715-ff12-41df-a9e7-58918a473190\") " pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:16.201467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.201435    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vccv2\" (UniqueName: \"kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2\") pod \"authorino-557f876f69-q57tr\" (UID: \"608d6715-ff12-41df-a9e7-58918a473190\") " pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:16.227182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.227151    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:16.227426 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.227411    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:16.252575 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.252553    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85d6df95fd-vwzns"]
Sep 10 16:15:16.255832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.255809    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:16.262294 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.262267    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85d6df95fd-vwzns"]
Sep 10 16:15:16.288940 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.288897    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85d6df95fd-vwzns"]
Sep 10 16:15:16.289292 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:15:16.289207    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dh9gj], unattached volumes=[], failed to process volumes=[kube-api-access-dh9gj]: context canceled" pod="kuadrant-system/authorino-85d6df95fd-vwzns" podUID="f5c357e4-7893-445f-8113-bb20885e427a"
Sep 10 16:15:16.390584 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.390555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj\") pod \"authorino-85d6df95fd-vwzns\" (UID: \"f5c357e4-7893-445f-8113-bb20885e427a\") " pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:16.491343 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.491259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj\") pod \"authorino-85d6df95fd-vwzns\" (UID: \"f5c357e4-7893-445f-8113-bb20885e427a\") " pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:16.504769 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.504746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj\") pod \"authorino-85d6df95fd-vwzns\" (UID: \"f5c357e4-7893-445f-8113-bb20885e427a\") " pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:16.513005 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:16.512985    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:16.515552 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:15:16.515529    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod608d6715_ff12_41df_a9e7_58918a473190.slice/crio-ea583034fd7d4ff360194364d3ca18b7af185dfe1f9746407644c3d058a02616 WatchSource:0}: Error finding container ea583034fd7d4ff360194364d3ca18b7af185dfe1f9746407644c3d058a02616: Status 404 returned error can't find the container with id ea583034fd7d4ff360194364d3ca18b7af185dfe1f9746407644c3d058a02616
Sep 10 16:15:17.299157 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.299109    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-q57tr" event={"ID":"608d6715-ff12-41df-a9e7-58918a473190","Type":"ContainerStarted","Data":"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"}
Sep 10 16:15:17.299157 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.299115    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-q57tr" podUID="608d6715-ff12-41df-a9e7-58918a473190" containerName="authorino" containerID="cri-o://0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372" gracePeriod=30
Sep 10 16:15:17.299675 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.299168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-q57tr" event={"ID":"608d6715-ff12-41df-a9e7-58918a473190","Type":"ContainerStarted","Data":"ea583034fd7d4ff360194364d3ca18b7af185dfe1f9746407644c3d058a02616"}
Sep 10 16:15:17.299675 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.299125    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:17.306778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.306757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:17.319729 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.319688    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-q57tr" podStartSLOduration=1.757671734 podStartE2EDuration="2.319674939s" totalImagesPullingTime="562.003205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:15:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:15:17.31906755 +0000 UTC m=+812.552015652" watchObservedRunningTime="2026-09-10 16:15:17.319674939 +0000 UTC m=+812.552623041"
Sep 10 16:15:17.398164 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.398093    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj\") pod \"f5c357e4-7893-445f-8113-bb20885e427a\" (UID: \"f5c357e4-7893-445f-8113-bb20885e427a\") "
Sep 10 16:15:17.400088 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.400056    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj" (OuterVolumeSpecName: "kube-api-access-dh9gj") pod "f5c357e4-7893-445f-8113-bb20885e427a" (UID: "f5c357e4-7893-445f-8113-bb20885e427a"). InnerVolumeSpecName "kube-api-access-dh9gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:15:17.498877 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.498850    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/f5c357e4-7893-445f-8113-bb20885e427a-kube-api-access-dh9gj\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:17.540973 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.540951    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:17.599467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.599443    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vccv2\" (UniqueName: \"kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2\") pod \"608d6715-ff12-41df-a9e7-58918a473190\" (UID: \"608d6715-ff12-41df-a9e7-58918a473190\") "
Sep 10 16:15:17.601318 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.601290    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2" (OuterVolumeSpecName: "kube-api-access-vccv2") pod "608d6715-ff12-41df-a9e7-58918a473190" (UID: "608d6715-ff12-41df-a9e7-58918a473190"). InnerVolumeSpecName "kube-api-access-vccv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:15:17.700134 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:17.700069    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vccv2\" (UniqueName: \"kubernetes.io/projected/608d6715-ff12-41df-a9e7-58918a473190-kube-api-access-vccv2\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:18.304751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304717    2574 generic.go:356] "Generic (PLEG): container finished" podID="608d6715-ff12-41df-a9e7-58918a473190" containerID="0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372" exitCode=0
Sep 10 16:15:18.304751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304761    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-q57tr"
Sep 10 16:15:18.305194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304778    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d6df95fd-vwzns"
Sep 10 16:15:18.305194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-q57tr" event={"ID":"608d6715-ff12-41df-a9e7-58918a473190","Type":"ContainerDied","Data":"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"}
Sep 10 16:15:18.305194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-q57tr" event={"ID":"608d6715-ff12-41df-a9e7-58918a473190","Type":"ContainerDied","Data":"ea583034fd7d4ff360194364d3ca18b7af185dfe1f9746407644c3d058a02616"}
Sep 10 16:15:18.305194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.304843    2574 scope.go:117] "RemoveContainer" containerID="0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"
Sep 10 16:15:18.313916 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.313894    2574 scope.go:117] "RemoveContainer" containerID="0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"
Sep 10 16:15:18.314218 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:15:18.314176    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372\": container with ID starting with 0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372 not found: ID does not exist" containerID="0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"
Sep 10 16:15:18.314287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.314213    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372"} err="failed to get container status \"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372\": rpc error: code = NotFound desc = could not find container \"0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372\": container with ID starting with 0a7c88aa1719546659f88b204379cef2276bcbd68778d68b1d69a52a7d2ef372 not found: ID does not exist"
Sep 10 16:15:18.345628 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.345600    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85d6df95fd-vwzns"]
Sep 10 16:15:18.348714 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.348690    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85d6df95fd-vwzns"]
Sep 10 16:15:18.363680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.363653    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:18.369675 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:18.369653    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-q57tr"]
Sep 10 16:15:19.265595 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.265564    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="608d6715-ff12-41df-a9e7-58918a473190" path="/var/lib/kubelet/pods/608d6715-ff12-41df-a9e7-58918a473190/volumes"
Sep 10 16:15:19.265873 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.265860    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5c357e4-7893-445f-8113-bb20885e427a" path="/var/lib/kubelet/pods/f5c357e4-7893-445f-8113-bb20885e427a/volumes"
Sep 10 16:15:19.442309 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.442275    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:15:19.442686 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.442458    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-7rpm6" podUID="807ad561-4096-417e-aeda-90ede3c1aaca" containerName="authorino" containerID="cri-o://a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17" gracePeriod=30
Sep 10 16:15:19.684079 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.684055    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:15:19.817138 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.817061    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rg4k\" (UniqueName: \"kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k\") pod \"807ad561-4096-417e-aeda-90ede3c1aaca\" (UID: \"807ad561-4096-417e-aeda-90ede3c1aaca\") "
Sep 10 16:15:19.819170 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.819144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k" (OuterVolumeSpecName: "kube-api-access-9rg4k") pod "807ad561-4096-417e-aeda-90ede3c1aaca" (UID: "807ad561-4096-417e-aeda-90ede3c1aaca"). InnerVolumeSpecName "kube-api-access-9rg4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:15:19.918285 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:19.918258    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rg4k\" (UniqueName: \"kubernetes.io/projected/807ad561-4096-417e-aeda-90ede3c1aaca-kube-api-access-9rg4k\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:20.314607 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.314570    2574 generic.go:356] "Generic (PLEG): container finished" podID="807ad561-4096-417e-aeda-90ede3c1aaca" containerID="a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17" exitCode=0
Sep 10 16:15:20.314771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.314611    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-7rpm6"
Sep 10 16:15:20.314771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.314654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-7rpm6" event={"ID":"807ad561-4096-417e-aeda-90ede3c1aaca","Type":"ContainerDied","Data":"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"}
Sep 10 16:15:20.314771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.314694    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-7rpm6" event={"ID":"807ad561-4096-417e-aeda-90ede3c1aaca","Type":"ContainerDied","Data":"34f7149a5bf9b5b296e56a7d2a229542481212be328c436b30858903172fdcab"}
Sep 10 16:15:20.314771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.314711    2574 scope.go:117] "RemoveContainer" containerID="a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"
Sep 10 16:15:20.322650 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.322629    2574 scope.go:117] "RemoveContainer" containerID="a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"
Sep 10 16:15:20.322923 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:15:20.322900    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17\": container with ID starting with a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17 not found: ID does not exist" containerID="a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"
Sep 10 16:15:20.322980 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.322936    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17"} err="failed to get container status \"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17\": rpc error: code = NotFound desc = could not find container \"a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17\": container with ID starting with a3222320bd43115df07f44a9233127e6721fc1789f591fc6608b99c368840c17 not found: ID does not exist"
Sep 10 16:15:20.340981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.340960    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:15:20.347653 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:20.347632    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-7rpm6"]
Sep 10 16:15:21.264910 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:21.264878    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="807ad561-4096-417e-aeda-90ede3c1aaca" path="/var/lib/kubelet/pods/807ad561-4096-417e-aeda-90ede3c1aaca/volumes"
Sep 10 16:15:25.472010 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.471974    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472234    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807ad561-4096-417e-aeda-90ede3c1aaca" containerName="authorino"
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472244    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="807ad561-4096-417e-aeda-90ede3c1aaca" containerName="authorino"
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472262    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="608d6715-ff12-41df-a9e7-58918a473190" containerName="authorino"
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472267    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="608d6715-ff12-41df-a9e7-58918a473190" containerName="authorino"
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472313    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="807ad561-4096-417e-aeda-90ede3c1aaca" containerName="authorino"
Sep 10 16:15:25.472377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.472322    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="608d6715-ff12-41df-a9e7-58918a473190" containerName="authorino"
Sep 10 16:15:25.477464 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.477442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.480947 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.480921    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:15:25.481116 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.481099    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:15:25.481454 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.481436    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:15:25.481538 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.481480    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-phs7m\""
Sep 10 16:15:25.492204 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.492184    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:25.557920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.557888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9cp\" (UniqueName: \"kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.558041 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.557931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.558041 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.557960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.658648 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.658619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9cp\" (UniqueName: \"kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.658817 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.658655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.658817 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.658682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.661136 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.661112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.661229 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.661155    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.674137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.674114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9cp\" (UniqueName: \"kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp\") pod \"maas-controller-6b4d59949b-n5nzs\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") " pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.787236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.787206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:25.841247 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.841204    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:15:25.844678 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.844653    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:25.859179 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.859118    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:15:25.961403 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.961375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:25.961566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.961426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:25.961566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:25.961464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4hk\" (UniqueName: \"kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.060426 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.060398    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:26.062114 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.062085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.062260 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.062139    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.062260 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.062167    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4hk\" (UniqueName: \"kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.062405 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:15:26.062342    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54911a6a_dc5e_4bda_a5b7_565aa627a0be.slice/crio-85e0ff70b41357b14eaada8b8919703733af79cc1617f6a1ea761967999d9449 WatchSource:0}: Error finding container 85e0ff70b41357b14eaada8b8919703733af79cc1617f6a1ea761967999d9449: Status 404 returned error can't find the container with id 85e0ff70b41357b14eaada8b8919703733af79cc1617f6a1ea761967999d9449
Sep 10 16:15:26.064611 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.064590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.064735 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.064720    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.071627 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.071607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4hk\" (UniqueName: \"kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk\") pod \"maas-controller-ccf7f868f-wx4kn\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") " pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.159064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.159037    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:26.345208 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.345129    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" event={"ID":"54911a6a-dc5e-4bda-a5b7-565aa627a0be","Type":"ContainerStarted","Data":"85e0ff70b41357b14eaada8b8919703733af79cc1617f6a1ea761967999d9449"}
Sep 10 16:15:26.430728 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:26.430704    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:15:26.432823 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:15:26.432800    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e208a8_cfe7_4bee_8092_e35c2847240f.slice/crio-3ec1bcf8d203a13994861ff31e974bb3f05bc95d432f22631336407c2ac304e2 WatchSource:0}: Error finding container 3ec1bcf8d203a13994861ff31e974bb3f05bc95d432f22631336407c2ac304e2: Status 404 returned error can't find the container with id 3ec1bcf8d203a13994861ff31e974bb3f05bc95d432f22631336407c2ac304e2
Sep 10 16:15:27.353586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:27.353535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" event={"ID":"33e208a8-cfe7-4bee-8092-e35c2847240f","Type":"ContainerStarted","Data":"3ec1bcf8d203a13994861ff31e974bb3f05bc95d432f22631336407c2ac304e2"}
Sep 10 16:15:29.364115 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.364079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" event={"ID":"54911a6a-dc5e-4bda-a5b7-565aa627a0be","Type":"ContainerStarted","Data":"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"}
Sep 10 16:15:29.364660 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.364219    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:29.365774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.365746    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" event={"ID":"33e208a8-cfe7-4bee-8092-e35c2847240f","Type":"ContainerStarted","Data":"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"}
Sep 10 16:15:29.365906 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.365816    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:29.390245 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.390203    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" podStartSLOduration=1.7985580909999999 podStartE2EDuration="4.390188979s" totalImagesPullingTime="2.591630888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:15:29.388657597 +0000 UTC m=+824.621605698" watchObservedRunningTime="2026-09-10 16:15:29.390188979 +0000 UTC m=+824.623137080"
Sep 10 16:15:29.410118 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:29.410071    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" podStartSLOduration=2.178160292 podStartE2EDuration="4.410058262s" totalImagesPullingTime="2.23189797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:15:29.409305623 +0000 UTC m=+824.642253725" watchObservedRunningTime="2026-09-10 16:15:29.410058262 +0000 UTC m=+824.643006412"
Sep 10 16:15:40.372767 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.372737    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:40.373292 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.372804    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:15:40.471087 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.471041    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:40.471416 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.471356    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" podUID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" containerName="manager" containerID="cri-o://9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923" gracePeriod=30
Sep 10 16:15:40.703832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.703803    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:40.879507 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.879474    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert\") pod \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") "
Sep 10 16:15:40.879680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.879513    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h9cp\" (UniqueName: \"kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp\") pod \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") "
Sep 10 16:15:40.879680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.879533    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs\") pod \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\" (UID: \"54911a6a-dc5e-4bda-a5b7-565aa627a0be\") "
Sep 10 16:15:40.881701 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.881671    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "54911a6a-dc5e-4bda-a5b7-565aa627a0be" (UID: "54911a6a-dc5e-4bda-a5b7-565aa627a0be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:15:40.881701 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.881690    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "54911a6a-dc5e-4bda-a5b7-565aa627a0be" (UID: "54911a6a-dc5e-4bda-a5b7-565aa627a0be"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:15:40.881891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.881690    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp" (OuterVolumeSpecName: "kube-api-access-9h9cp") pod "54911a6a-dc5e-4bda-a5b7-565aa627a0be" (UID: "54911a6a-dc5e-4bda-a5b7-565aa627a0be"). InnerVolumeSpecName "kube-api-access-9h9cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:15:40.980776 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.980748    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-webhook-cert\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:40.980776 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.980773    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h9cp\" (UniqueName: \"kubernetes.io/projected/54911a6a-dc5e-4bda-a5b7-565aa627a0be-kube-api-access-9h9cp\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:40.980931 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:40.980782    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54911a6a-dc5e-4bda-a5b7-565aa627a0be-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:15:41.414773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.414733    2574 generic.go:356] "Generic (PLEG): container finished" podID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" containerID="9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923" exitCode=0
Sep 10 16:15:41.414773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.414773    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b4d59949b-n5nzs"
Sep 10 16:15:41.415206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.414821    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" event={"ID":"54911a6a-dc5e-4bda-a5b7-565aa627a0be","Type":"ContainerDied","Data":"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"}
Sep 10 16:15:41.415206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.414874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b4d59949b-n5nzs" event={"ID":"54911a6a-dc5e-4bda-a5b7-565aa627a0be","Type":"ContainerDied","Data":"85e0ff70b41357b14eaada8b8919703733af79cc1617f6a1ea761967999d9449"}
Sep 10 16:15:41.415206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.414898    2574 scope.go:117] "RemoveContainer" containerID="9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"
Sep 10 16:15:41.423024 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.423007    2574 scope.go:117] "RemoveContainer" containerID="9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"
Sep 10 16:15:41.423302 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:15:41.423276    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923\": container with ID starting with 9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923 not found: ID does not exist" containerID="9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"
Sep 10 16:15:41.423398 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.423310    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923"} err="failed to get container status \"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923\": rpc error: code = NotFound desc = could not find container \"9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923\": container with ID starting with 9ce8ac9eb04cad6279eb1715aa1d1d4d0c51d94ca4d27bce4e067bb8078fe923 not found: ID does not exist"
Sep 10 16:15:41.435858 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.435835    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:41.440876 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:41.440854    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b4d59949b-n5nzs"]
Sep 10 16:15:43.266007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:15:43.265972    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" path="/var/lib/kubelet/pods/54911a6a-dc5e-4bda-a5b7-565aa627a0be/volumes"
Sep 10 16:16:01.776856 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.776822    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:16:01.777343 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.777094    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" containerName="manager"
Sep 10 16:16:01.777343 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.777105    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" containerName="manager"
Sep 10 16:16:01.777343 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.777151    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="54911a6a-dc5e-4bda-a5b7-565aa627a0be" containerName="manager"
Sep 10 16:16:01.782318 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.782299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.785358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.785322    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 16:16:01.785774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.785758    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 16:16:01.786672 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.786653    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2l6fb\""
Sep 10 16:16:01.786762 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.786695    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 16:16:01.786887 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.786808    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 16:16:01.792478 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.792456    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:16:01.834424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.834403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.834539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.834467    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.834585 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.834536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx96n\" (UniqueName: \"kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.834585 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.834562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.935265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.935231    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.935440 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.935275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx96n\" (UniqueName: \"kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.935440 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.935294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.935440 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.935314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.935440 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:16:01.935400    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 16:16:01.935629 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:16:01.935465    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs podName:d5f70685-6550-4288-ab90-90b452b7f9ed nodeName:}" failed. No retries permitted until 2026-09-10 16:16:02.435449369 +0000 UTC m=+857.668397471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs") pod "maas-api-b8859f477-mwplc" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed") : secret "maas-api-metrics-tls" not found
Sep 10 16:16:01.935629 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:16:01.935407    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 10 16:16:01.935629 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:16:01.935543    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls podName:d5f70685-6550-4288-ab90-90b452b7f9ed nodeName:}" failed. No retries permitted until 2026-09-10 16:16:02.435527413 +0000 UTC m=+857.668475509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls") pod "maas-api-b8859f477-mwplc" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed") : secret "maas-api-serving-cert" not found
Sep 10 16:16:01.935898 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.935882    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:01.954781 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:01.954757    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx96n\" (UniqueName: \"kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.439531 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.439499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.439688 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.439549    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.441708 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.441683    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.441820 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.441716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") pod \"maas-api-b8859f477-mwplc\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") " pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.692663 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.692579    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:02.971668 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:02.971588    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:16:02.975349 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:16:02.975303    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f70685_6550_4288_ab90_90b452b7f9ed.slice/crio-426af405615bbeaa365d64e01b8b4ed7caa97b7138043b82c119da8c63516f46 WatchSource:0}: Error finding container 426af405615bbeaa365d64e01b8b4ed7caa97b7138043b82c119da8c63516f46: Status 404 returned error can't find the container with id 426af405615bbeaa365d64e01b8b4ed7caa97b7138043b82c119da8c63516f46
Sep 10 16:16:03.509424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:03.509385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" event={"ID":"d5f70685-6550-4288-ab90-90b452b7f9ed","Type":"ContainerStarted","Data":"426af405615bbeaa365d64e01b8b4ed7caa97b7138043b82c119da8c63516f46"}
Sep 10 16:16:05.520410 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:05.520375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" event={"ID":"d5f70685-6550-4288-ab90-90b452b7f9ed","Type":"ContainerStarted","Data":"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"}
Sep 10 16:16:05.520793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:05.520494    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:05.544566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:05.544520    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" podStartSLOduration=2.778276733 podStartE2EDuration="4.544508819s" totalImagesPullingTime="1.766232086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:16:05.54270349 +0000 UTC m=+860.775651605" watchObservedRunningTime="2026-09-10 16:16:05.544508819 +0000 UTC m=+860.777456924"
Sep 10 16:16:11.529383 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:11.529343    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:16:45.224441 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:45.224370    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:16:45.225220 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:45.225198    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:16:47.344972 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.344939    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"]
Sep 10 16:16:47.348026 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.348007    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.372471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.372443    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 16:16:47.373723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.373680    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wz694\""
Sep 10 16:16:47.373723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.373701    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 16:16:47.373922 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.373743    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 16:16:47.386690 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.386655    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"]
Sep 10 16:16:47.504419 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ffj8\" (UniqueName: \"kubernetes.io/projected/321f74f0-b992-46d6-a8a6-4091350db6e8-kube-api-access-6ffj8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.504601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.504601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504465    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.504601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.504759 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.504759 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.504677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321f74f0-b992-46d6-a8a6-4091350db6e8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605534 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605534 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605487    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605534 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321f74f0-b992-46d6-a8a6-4091350db6e8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ffj8\" (UniqueName: \"kubernetes.io/projected/321f74f0-b992-46d6-a8a6-4091350db6e8-kube-api-access-6ffj8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605937 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605856    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.605937 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.606037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.605945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.607774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.607748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/321f74f0-b992-46d6-a8a6-4091350db6e8-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.607973 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.607953    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/321f74f0-b992-46d6-a8a6-4091350db6e8-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.636622 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.636590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ffj8\" (UniqueName: \"kubernetes.io/projected/321f74f0-b992-46d6-a8a6-4091350db6e8-kube-api-access-6ffj8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r\" (UID: \"321f74f0-b992-46d6-a8a6-4091350db6e8\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.657202 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.657176    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:47.955801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.955721    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"]
Sep 10 16:16:47.960164 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:16:47.960130    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321f74f0_b992_46d6_a8a6_4091350db6e8.slice/crio-22d7bd80215e3d00a285a412ad3859e3b702baa0044511b607af6914f04d85a5 WatchSource:0}: Error finding container 22d7bd80215e3d00a285a412ad3859e3b702baa0044511b607af6914f04d85a5: Status 404 returned error can't find the container with id 22d7bd80215e3d00a285a412ad3859e3b702baa0044511b607af6914f04d85a5
Sep 10 16:16:47.961530 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:47.961513    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:16:48.701895 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:48.701856    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r" event={"ID":"321f74f0-b992-46d6-a8a6-4091350db6e8","Type":"ContainerStarted","Data":"22d7bd80215e3d00a285a412ad3859e3b702baa0044511b607af6914f04d85a5"}
Sep 10 16:16:49.707066 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:49.707026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r" event={"ID":"321f74f0-b992-46d6-a8a6-4091350db6e8","Type":"ContainerStarted","Data":"59bbb256824d626be17cee3f2a4dd3244dbcfb240ce523b36d89374988080b98"}
Sep 10 16:16:49.746993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:49.746936    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r" podStartSLOduration=1.738732178 podStartE2EDuration="2.746919695s" totalImagesPullingTime="1.008187517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:16:49.745697612 +0000 UTC m=+904.978645713" watchObservedRunningTime="2026-09-10 16:16:49.746919695 +0000 UTC m=+904.979867796"
Sep 10 16:16:57.305467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.305431    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"]
Sep 10 16:16:57.308869 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.308850    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.311707 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.311685    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 16:16:57.321064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.321041    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"]
Sep 10 16:16:57.380211 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsct6\" (UniqueName: \"kubernetes.io/projected/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-kube-api-access-gsct6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.380211 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.380435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.380435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380303    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.380435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.380535 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.380464    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.480925 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.480889    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481110 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.480947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481110 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.480973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481110 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.480999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsct6\" (UniqueName: \"kubernetes.io/projected/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-kube-api-access-gsct6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481110 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.481016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481110 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.481035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481425 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.481385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.481432    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.481491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.481476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.483407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.483383    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.483590 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.483570    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.502987 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.502953    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsct6\" (UniqueName: \"kubernetes.io/projected/794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13-kube-api-access-gsct6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr\" (UID: \"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.618609 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.618518    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:16:57.657744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.657714    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:57.657890 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.657778    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:57.673746 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.673708    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:57.750711 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.750683    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r"
Sep 10 16:16:57.894033 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:57.894008    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"]
Sep 10 16:16:57.896664 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:16:57.896629    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794ac9e1_a70f_4c1d_b1d1_3e3ce0a7fa13.slice/crio-86fdfe1eaf6d69cac6ab0ddce6131fe9db6a6eb459b99f0bad4593147a8480fa WatchSource:0}: Error finding container 86fdfe1eaf6d69cac6ab0ddce6131fe9db6a6eb459b99f0bad4593147a8480fa: Status 404 returned error can't find the container with id 86fdfe1eaf6d69cac6ab0ddce6131fe9db6a6eb459b99f0bad4593147a8480fa
Sep 10 16:16:58.744389 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:58.744353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr" event={"ID":"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13","Type":"ContainerStarted","Data":"0fc2fa6f7a33c656daae8382f0c38475cfaafb68eb8ac377cde0c1c9407801ac"}
Sep 10 16:16:58.744776 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:16:58.744417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr" event={"ID":"794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13","Type":"ContainerStarted","Data":"86fdfe1eaf6d69cac6ab0ddce6131fe9db6a6eb459b99f0bad4593147a8480fa"}
Sep 10 16:17:07.619483 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.619432    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:17:07.619931 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.619516    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:17:07.632150 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.632117    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:17:07.665188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.665091    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr" podStartSLOduration=10.460369245 podStartE2EDuration="10.665070185s" totalImagesPullingTime="204.70094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:16:58.776668212 +0000 UTC m=+914.009616313" watchObservedRunningTime="2026-09-10 16:17:07.665070185 +0000 UTC m=+922.898018292"
Sep 10 16:17:07.791842 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.791809    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr"
Sep 10 16:17:07.870819 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.870732    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"]
Sep 10 16:17:07.874206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.874183    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.885756 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.885732    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 16:17:07.906373 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.906323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"]
Sep 10 16:17:07.966796 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.966981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.966981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.966981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.966981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966946    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16db3c28-5298-4baa-bfd9-2905cfbde717-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:07.966981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:07.966974    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9jh8\" (UniqueName: \"kubernetes.io/projected/16db3c28-5298-4baa-bfd9-2905cfbde717-kube-api-access-w9jh8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067442 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067397    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16db3c28-5298-4baa-bfd9-2905cfbde717-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067561    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9jh8\" (UniqueName: \"kubernetes.io/projected/16db3c28-5298-4baa-bfd9-2905cfbde717-kube-api-access-w9jh8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067968 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.067968 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067882    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.068034 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.067968    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.069835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.069814    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16db3c28-5298-4baa-bfd9-2905cfbde717-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.070137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.070116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16db3c28-5298-4baa-bfd9-2905cfbde717-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.079614 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.079587    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9jh8\" (UniqueName: \"kubernetes.io/projected/16db3c28-5298-4baa-bfd9-2905cfbde717-kube-api-access-w9jh8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk\" (UID: \"16db3c28-5298-4baa-bfd9-2905cfbde717\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.183567 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.183479    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:08.462647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.462600    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"]
Sep 10 16:17:08.464888 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:17:08.464863    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16db3c28_5298_4baa_bfd9_2905cfbde717.slice/crio-6a2a15514d567e2c1ce9df351f1d7d8608d38d0ee42f75d0ee8b83f94395b779 WatchSource:0}: Error finding container 6a2a15514d567e2c1ce9df351f1d7d8608d38d0ee42f75d0ee8b83f94395b779: Status 404 returned error can't find the container with id 6a2a15514d567e2c1ce9df351f1d7d8608d38d0ee42f75d0ee8b83f94395b779
Sep 10 16:17:08.786723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.786684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk" event={"ID":"16db3c28-5298-4baa-bfd9-2905cfbde717","Type":"ContainerStarted","Data":"9fcc4decd5320aefae21fa2b6fe5c450afd53285acdd26d6584cdbd13920b535"}
Sep 10 16:17:08.787094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.786737    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk" event={"ID":"16db3c28-5298-4baa-bfd9-2905cfbde717","Type":"ContainerStarted","Data":"6a2a15514d567e2c1ce9df351f1d7d8608d38d0ee42f75d0ee8b83f94395b779"}
Sep 10 16:17:08.822048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:08.821983    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk" podStartSLOduration=1.614047855 podStartE2EDuration="1.821963534s" totalImagesPullingTime="207.915679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:08.818190109 +0000 UTC m=+924.051138210" watchObservedRunningTime="2026-09-10 16:17:08.821963534 +0000 UTC m=+924.054911636"
Sep 10 16:17:18.184014 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:18.183978    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:18.184576 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:18.184122    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:18.196987 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:18.196963    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:17:18.841850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:17:18.841814    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk"
Sep 10 16:18:30.477101 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.477065    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-854f6b5cfc-krm9b"]
Sep 10 16:18:30.480309 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.480290    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.485673 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.485649    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h5k2r\""
Sep 10 16:18:30.485777 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.485676    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 16:18:30.495291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.495266    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-854f6b5cfc-krm9b"]
Sep 10 16:18:30.576063 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.576034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7wcr\" (UniqueName: \"kubernetes.io/projected/57751bf1-9b64-4657-9a71-27aafda7848e-kube-api-access-r7wcr\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.576172 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.576092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57751bf1-9b64-4657-9a71-27aafda7848e-tls-cert\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.677496 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.677462    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57751bf1-9b64-4657-9a71-27aafda7848e-tls-cert\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.677632 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.677537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7wcr\" (UniqueName: \"kubernetes.io/projected/57751bf1-9b64-4657-9a71-27aafda7848e-kube-api-access-r7wcr\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.679928 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.679903    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/57751bf1-9b64-4657-9a71-27aafda7848e-tls-cert\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.687552 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.687530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7wcr\" (UniqueName: \"kubernetes.io/projected/57751bf1-9b64-4657-9a71-27aafda7848e-kube-api-access-r7wcr\") pod \"authorino-854f6b5cfc-krm9b\" (UID: \"57751bf1-9b64-4657-9a71-27aafda7848e\") " pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:30.788683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:30.788656    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854f6b5cfc-krm9b"
Sep 10 16:18:31.056862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:31.056834    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-854f6b5cfc-krm9b"]
Sep 10 16:18:31.059667 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:18:31.059639    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57751bf1_9b64_4657_9a71_27aafda7848e.slice/crio-6c30a33790665d88661468782bf5a6a4391a21d289d1019998cb70f6896b13d3 WatchSource:0}: Error finding container 6c30a33790665d88661468782bf5a6a4391a21d289d1019998cb70f6896b13d3: Status 404 returned error can't find the container with id 6c30a33790665d88661468782bf5a6a4391a21d289d1019998cb70f6896b13d3
Sep 10 16:18:31.139070 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:31.139039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-854f6b5cfc-krm9b" event={"ID":"57751bf1-9b64-4657-9a71-27aafda7848e","Type":"ContainerStarted","Data":"6c30a33790665d88661468782bf5a6a4391a21d289d1019998cb70f6896b13d3"}
Sep 10 16:18:32.146588 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:32.146500    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-854f6b5cfc-krm9b" event={"ID":"57751bf1-9b64-4657-9a71-27aafda7848e","Type":"ContainerStarted","Data":"1b55622a7ad3775930db64b46749ded34a1521a1dbc867b5c0a473dcf5947639"}
Sep 10 16:18:32.176316 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:32.176260    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-854f6b5cfc-krm9b" podStartSLOduration=1.400178263 podStartE2EDuration="2.176243977s" totalImagesPullingTime="776.065714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:32.173599994 +0000 UTC m=+1007.406548097" watchObservedRunningTime="2026-09-10 16:18:32.176243977 +0000 UTC m=+1007.409192134"
Sep 10 16:18:54.616370 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.616315    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:18:54.619908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.619884    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.623165 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.623143    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-37c805-serving-cert\""
Sep 10 16:18:54.637835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.636573    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:18:54.679768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.679733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.680022 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.680001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.680149 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.680134    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl7mv\" (UniqueName: \"kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.680297 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.680283    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.781880 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.781843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.782050 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.781888    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.782050 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.781920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7mv\" (UniqueName: \"kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.782050 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.781953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.782553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.782526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.784371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.784341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.784463 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.784414    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.793187 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.793166    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl7mv\" (UniqueName: \"kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv\") pod \"maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:54.931282 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:54.931206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:55.209177 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:55.209101    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:18:55.212925 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:18:55.212895    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae45a17_c19f_4319_b917_48385e06c6a3.slice/crio-a6cee2602596d81e9dc97c300ce8c97936adb63aecaafd7bd7fe79cb4b001b59 WatchSource:0}: Error finding container a6cee2602596d81e9dc97c300ce8c97936adb63aecaafd7bd7fe79cb4b001b59: Status 404 returned error can't find the container with id a6cee2602596d81e9dc97c300ce8c97936adb63aecaafd7bd7fe79cb4b001b59
Sep 10 16:18:55.247708 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:55.247679    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" event={"ID":"aae45a17-c19f-4319-b917-48385e06c6a3","Type":"ContainerStarted","Data":"a6cee2602596d81e9dc97c300ce8c97936adb63aecaafd7bd7fe79cb4b001b59"}
Sep 10 16:18:56.253417 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:56.253378    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" event={"ID":"aae45a17-c19f-4319-b917-48385e06c6a3","Type":"ContainerStarted","Data":"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"}
Sep 10 16:18:56.253833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:56.253515    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:18:56.276981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:18:56.276933    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" podStartSLOduration=1.834075391 podStartE2EDuration="2.276917594s" totalImagesPullingTime="442.842203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:56.276006789 +0000 UTC m=+1031.508954894" watchObservedRunningTime="2026-09-10 16:18:56.276917594 +0000 UTC m=+1031.509865696"
Sep 10 16:19:02.265491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:02.265454    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:19:03.681309 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.681273    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:19:03.685396 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.685379    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.688315 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.688292    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-1596ac-serving-cert\""
Sep 10 16:19:03.698647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.698625    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:19:03.760865 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.760834    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjns4\" (UniqueName: \"kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.761046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.760876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.761046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.760898    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.761046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.760932    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.861564 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.861533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjns4\" (UniqueName: \"kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.861729 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.861577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.861729 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.861598    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.861729 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:03.861683    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-serving-cert: secret "maas-api-e2e-autores-1596ac-serving-cert" not found
Sep 10 16:19:03.861898 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.861721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.861898 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:03.861745    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls podName:bb673278-7629-4222-b91b-f14c519a5ae5 nodeName:}" failed. No retries permitted until 2026-09-10 16:19:04.361727967 +0000 UTC m=+1039.594676047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls") pod "maas-api-e2e-autores-1596ac-746f647786-765mj" (UID: "bb673278-7629-4222-b91b-f14c519a5ae5") : secret "maas-api-e2e-autores-1596ac-serving-cert" not found
Sep 10 16:19:03.862350 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.862306    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.863994 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.863977    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:03.877348 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:03.877312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjns4\" (UniqueName: \"kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:04.366635 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.366593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:04.369280 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.369253    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") pod \"maas-api-e2e-autores-1596ac-746f647786-765mj\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:04.596716 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.596681    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:04.806220 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.806186    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:04.809584 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.809570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.812076 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.812057    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fefaa7ab-serving-cert\""
Sep 10 16:19:04.819862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.819836    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:04.871013 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.870982    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn87j\" (UniqueName: \"kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.871146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.871041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.871146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.871078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.871146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.871096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.877893 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.877835    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:19:04.880127 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:19:04.880095    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb673278_7629_4222_b91b_f14c519a5ae5.slice/crio-d679b1dcf554f1a5993efe901a717d7d900a80ec3af257ef2692385d2569785b WatchSource:0}: Error finding container d679b1dcf554f1a5993efe901a717d7d900a80ec3af257ef2692385d2569785b: Status 404 returned error can't find the container with id d679b1dcf554f1a5993efe901a717d7d900a80ec3af257ef2692385d2569785b
Sep 10 16:19:04.972346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.972257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.972346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.972291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.972346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.972313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.972547 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.972387    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn87j\" (UniqueName: \"kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.972547 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:04.972415    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-serving-cert: secret "maas-api-e2e-ait-fefaa7ab-serving-cert" not found
Sep 10 16:19:04.972547 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:04.972491    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls podName:ee765225-8446-4e9a-87bc-232ecf783f03 nodeName:}" failed. No retries permitted until 2026-09-10 16:19:05.472465249 +0000 UTC m=+1040.705413329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls") pod "maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" (UID: "ee765225-8446-4e9a-87bc-232ecf783f03") : secret "maas-api-e2e-ait-fefaa7ab-serving-cert" not found
Sep 10 16:19:04.973049 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.973031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.974684 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.974663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:04.984477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:04.984452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn87j\" (UniqueName: \"kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:05.294969 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.294931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" event={"ID":"bb673278-7629-4222-b91b-f14c519a5ae5","Type":"ContainerStarted","Data":"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"}
Sep 10 16:19:05.295111 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.294996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" event={"ID":"bb673278-7629-4222-b91b-f14c519a5ae5","Type":"ContainerStarted","Data":"d679b1dcf554f1a5993efe901a717d7d900a80ec3af257ef2692385d2569785b"}
Sep 10 16:19:05.295111 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.295046    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:05.315463 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.315415    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" podStartSLOduration=2.000539127 podStartE2EDuration="2.315401544s" totalImagesPullingTime="314.862417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:05.313725193 +0000 UTC m=+1040.546673296" watchObservedRunningTime="2026-09-10 16:19:05.315401544 +0000 UTC m=+1040.548349644"
Sep 10 16:19:05.477604 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.477568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:05.480203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.480170    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") pod \"maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:05.721801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.721768    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:05.997785 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:05.997760    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:06.000070 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:19:06.000029    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee765225_8446_4e9a_87bc_232ecf783f03.slice/crio-9f98f5555b8c60c056b6c6e898e5e778589293efeb79a4dfddc423094897b77a WatchSource:0}: Error finding container 9f98f5555b8c60c056b6c6e898e5e778589293efeb79a4dfddc423094897b77a: Status 404 returned error can't find the container with id 9f98f5555b8c60c056b6c6e898e5e778589293efeb79a4dfddc423094897b77a
Sep 10 16:19:06.300886 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:06.300846    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" event={"ID":"ee765225-8446-4e9a-87bc-232ecf783f03","Type":"ContainerStarted","Data":"9f98f5555b8c60c056b6c6e898e5e778589293efeb79a4dfddc423094897b77a"}
Sep 10 16:19:07.307102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:07.307067    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" event={"ID":"ee765225-8446-4e9a-87bc-232ecf783f03","Type":"ContainerStarted","Data":"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"}
Sep 10 16:19:07.307571 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:07.307171    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:07.336367 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:07.336304    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" podStartSLOduration=2.988015499 podStartE2EDuration="3.336291507s" totalImagesPullingTime="348.276008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:07.334918727 +0000 UTC m=+1042.567866829" watchObservedRunningTime="2026-09-10 16:19:07.336291507 +0000 UTC m=+1042.569239609"
Sep 10 16:19:08.599219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.599185    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:19:08.602752 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.602732    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.605810 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.605787    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-aa8ef0bd-kserve-self-signed-certs\""
Sep 10 16:19:08.613950 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.613926    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:19:08.702062 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.702194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.702194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.702265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.702265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.702366 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.702293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp6zp\" (UniqueName: \"kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803581 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803547    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803713 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803605    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803713 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803639    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803713 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803856 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.803908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.803850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp6zp\" (UniqueName: \"kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.804086 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.804064    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.804168 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.804101    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.804226 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.804180    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.805924 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.805901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.806052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.806034    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.813802 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.813779    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp6zp\" (UniqueName: \"kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp\") pod \"e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") " pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:08.910851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:08.910794    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:09.251885 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:19:09.251851    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50cbb9a9_42ac_4274_8d20_6227e4999b4c.slice/crio-51373b87b55b9a2dc560135b10c574df60889988363c3d66239650aff00b87e4 WatchSource:0}: Error finding container 51373b87b55b9a2dc560135b10c574df60889988363c3d66239650aff00b87e4: Status 404 returned error can't find the container with id 51373b87b55b9a2dc560135b10c574df60889988363c3d66239650aff00b87e4
Sep 10 16:19:09.267001 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:09.266978    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:19:09.318561 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:09.318529    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" event={"ID":"50cbb9a9-42ac-4274-8d20-6227e4999b4c","Type":"ContainerStarted","Data":"51373b87b55b9a2dc560135b10c574df60889988363c3d66239650aff00b87e4"}
Sep 10 16:19:10.329691 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:10.329650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" event={"ID":"50cbb9a9-42ac-4274-8d20-6227e4999b4c","Type":"ContainerStarted","Data":"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"}
Sep 10 16:19:10.356446 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:10.356385    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" podStartSLOduration=2.356364388 podStartE2EDuration="2.356364388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:10.353110362 +0000 UTC m=+1045.586058475" watchObservedRunningTime="2026-09-10 16:19:10.356364388 +0000 UTC m=+1045.589312490"
Sep 10 16:19:11.306099 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:11.306071    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:19:13.316528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:13.316496    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:18.911032 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:18.910995    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:18.911533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:18.911049    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:18.924377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:18.924347    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:19.382565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:19.382538    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:19:23.999712 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:23.999671    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:19:24.003566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.003547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.006513 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.006473    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e22c72-serving-cert\""
Sep 10 16:19:24.015125 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.015097    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:19:24.051857 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.051825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.051992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.051870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.051992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.051934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qc52\" (UniqueName: \"kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.051992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.051981    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.153427 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.153386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.153596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.153443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qc52\" (UniqueName: \"kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.153596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.153471    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.153596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.153543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.153715 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:24.153655    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-serving-cert: secret "maas-api-e2e-inf-b-e22c72-serving-cert" not found
Sep 10 16:19:24.153754 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:24.153716    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls podName:db85115f-cc59-4931-b5f3-43b84588c43e nodeName:}" failed. No retries permitted until 2026-09-10 16:19:24.653698051 +0000 UTC m=+1059.886646131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls") pod "maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e") : secret "maas-api-e2e-inf-b-e22c72-serving-cert" not found
Sep 10 16:19:24.154233 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.154209    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.156038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.156011    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.169096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.169073    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qc52\" (UniqueName: \"kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.657695 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:24.657661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:24.657879 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:24.657839    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-serving-cert: secret "maas-api-e2e-inf-b-e22c72-serving-cert" not found
Sep 10 16:19:24.657951 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:24.657922    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls podName:db85115f-cc59-4931-b5f3-43b84588c43e nodeName:}" failed. No retries permitted until 2026-09-10 16:19:25.657898423 +0000 UTC m=+1060.890846527 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls") pod "maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e") : secret "maas-api-e2e-inf-b-e22c72-serving-cert" not found
Sep 10 16:19:25.666655 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:25.666619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:25.669111 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:25.669087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:25.815908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:25.815871    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:26.089164 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:26.089130    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:19:26.092262 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:19:26.092228    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb85115f_cc59_4931_b5f3_43b84588c43e.slice/crio-eaf64903003b4eb6570dc54a6fab143a6a2ed000d22129565e51538830882668 WatchSource:0}: Error finding container eaf64903003b4eb6570dc54a6fab143a6a2ed000d22129565e51538830882668: Status 404 returned error can't find the container with id eaf64903003b4eb6570dc54a6fab143a6a2ed000d22129565e51538830882668
Sep 10 16:19:26.405789 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:26.405712    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" event={"ID":"db85115f-cc59-4931-b5f3-43b84588c43e","Type":"ContainerStarted","Data":"eaf64903003b4eb6570dc54a6fab143a6a2ed000d22129565e51538830882668"}
Sep 10 16:19:27.412249 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:27.412213    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" event={"ID":"db85115f-cc59-4931-b5f3-43b84588c43e","Type":"ContainerStarted","Data":"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"}
Sep 10 16:19:27.412642 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:27.412319    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:27.438156 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:27.438103    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" podStartSLOduration=3.877638121 podStartE2EDuration="4.438088479s" totalImagesPullingTime="560.450358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:27.435843904 +0000 UTC m=+1062.668792006" watchObservedRunningTime="2026-09-10 16:19:27.438088479 +0000 UTC m=+1062.671036581"
Sep 10 16:19:33.423370 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:33.423319    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:19:45.820537 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:45.820505    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:45.820924 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:45.820706    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" podUID="ee765225-8446-4e9a-87bc-232ecf783f03" containerName="maas-api" containerID="cri-o://229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5" gracePeriod=30
Sep 10 16:19:46.072060 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.072005    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:46.150995 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.150965    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs\") pod \"ee765225-8446-4e9a-87bc-232ecf783f03\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") "
Sep 10 16:19:46.151186 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.151015    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn87j\" (UniqueName: \"kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j\") pod \"ee765225-8446-4e9a-87bc-232ecf783f03\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") "
Sep 10 16:19:46.151186 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.151043    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") pod \"ee765225-8446-4e9a-87bc-232ecf783f03\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") "
Sep 10 16:19:46.151186 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.151087    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca\") pod \"ee765225-8446-4e9a-87bc-232ecf783f03\" (UID: \"ee765225-8446-4e9a-87bc-232ecf783f03\") "
Sep 10 16:19:46.151543 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.151517    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee765225-8446-4e9a-87bc-232ecf783f03" (UID: "ee765225-8446-4e9a-87bc-232ecf783f03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:46.153189 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.153156    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee765225-8446-4e9a-87bc-232ecf783f03" (UID: "ee765225-8446-4e9a-87bc-232ecf783f03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:46.153269 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.153232    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee765225-8446-4e9a-87bc-232ecf783f03" (UID: "ee765225-8446-4e9a-87bc-232ecf783f03"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:46.153269 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.153237    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j" (OuterVolumeSpecName: "kube-api-access-jn87j") pod "ee765225-8446-4e9a-87bc-232ecf783f03" (UID: "ee765225-8446-4e9a-87bc-232ecf783f03"). InnerVolumeSpecName "kube-api-access-jn87j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:19:46.252284 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.252252    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:19:46.252284 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.252278    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn87j\" (UniqueName: \"kubernetes.io/projected/ee765225-8446-4e9a-87bc-232ecf783f03-kube-api-access-jn87j\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:19:46.252284 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.252288    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee765225-8446-4e9a-87bc-232ecf783f03-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:19:46.252522 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.252298    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee765225-8446-4e9a-87bc-232ecf783f03-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:19:46.503188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.503099    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee765225-8446-4e9a-87bc-232ecf783f03" containerID="229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5" exitCode=0
Sep 10 16:19:46.503188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.503144    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"
Sep 10 16:19:46.503188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.503179    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" event={"ID":"ee765225-8446-4e9a-87bc-232ecf783f03","Type":"ContainerDied","Data":"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"}
Sep 10 16:19:46.503460 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.503231    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc" event={"ID":"ee765225-8446-4e9a-87bc-232ecf783f03","Type":"ContainerDied","Data":"9f98f5555b8c60c056b6c6e898e5e778589293efeb79a4dfddc423094897b77a"}
Sep 10 16:19:46.503460 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.503252    2574 scope.go:117] "RemoveContainer" containerID="229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"
Sep 10 16:19:46.512655 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.512639    2574 scope.go:117] "RemoveContainer" containerID="229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"
Sep 10 16:19:46.512928 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:46.512911    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5\": container with ID starting with 229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5 not found: ID does not exist" containerID="229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"
Sep 10 16:19:46.512997 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.512933    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5"} err="failed to get container status \"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5\": rpc error: code = NotFound desc = could not find container \"229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5\": container with ID starting with 229478fcef02178ea0ff1cd849184b1ff8ec835bf2d12ad8d0cf34c560f3d4a5 not found: ID does not exist"
Sep 10 16:19:46.532796 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.532774    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:46.536182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:46.536160    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fefaa7ab-5cbd9cf44f-5pmpc"]
Sep 10 16:19:47.267007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:47.266971    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee765225-8446-4e9a-87bc-232ecf783f03" path="/var/lib/kubelet/pods/ee765225-8446-4e9a-87bc-232ecf783f03/volumes"
Sep 10 16:19:50.528067 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.528026    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:19:50.528476 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.528417    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee765225-8446-4e9a-87bc-232ecf783f03" containerName="maas-api"
Sep 10 16:19:50.528476 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.528436    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee765225-8446-4e9a-87bc-232ecf783f03" containerName="maas-api"
Sep 10 16:19:50.528573 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.528559    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee765225-8446-4e9a-87bc-232ecf783f03" containerName="maas-api"
Sep 10 16:19:50.532680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.532661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.536003 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.535981    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b7764c7d-serving-cert\""
Sep 10 16:19:50.543787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.543765    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:19:50.690745 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.690711    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.690745 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.690747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.690945 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.690798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.690945 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.690830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r474f\" (UniqueName: \"kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.792094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.792010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.792094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.792076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.792272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.792140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.792272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.792194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r474f\" (UniqueName: \"kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.792387 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:50.792300    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-serving-cert: secret "maas-api-e2e-migrate-b7764c7d-serving-cert" not found
Sep 10 16:19:50.792423 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:19:50.792397    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls podName:9cf14a3b-d25c-4d9a-b210-8088b003e50d nodeName:}" failed. No retries permitted until 2026-09-10 16:19:51.292375151 +0000 UTC m=+1086.525323231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls") pod "maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" (UID: "9cf14a3b-d25c-4d9a-b210-8088b003e50d") : secret "maas-api-e2e-migrate-b7764c7d-serving-cert" not found
Sep 10 16:19:50.792671 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.792655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.794600 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.794579    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:50.802880 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:50.802858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r474f\" (UniqueName: \"kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:51.296984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:51.296941    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:51.299321 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:51.299286    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") pod \"maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:51.443946 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:51.443910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:51.719909 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:51.719878    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:19:51.721995 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:19:51.721973    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf14a3b_d25c_4d9a_b210_8088b003e50d.slice/crio-a01ad91a9ea5ee6dfd550e0182f8958302f6c03828906276381650dfc8ef2609 WatchSource:0}: Error finding container a01ad91a9ea5ee6dfd550e0182f8958302f6c03828906276381650dfc8ef2609: Status 404 returned error can't find the container with id a01ad91a9ea5ee6dfd550e0182f8958302f6c03828906276381650dfc8ef2609
Sep 10 16:19:52.538778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:52.538738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" event={"ID":"9cf14a3b-d25c-4d9a-b210-8088b003e50d","Type":"ContainerStarted","Data":"879372329c13451b36f72521a6dfb0c23ce2def429455f21480d47f7f19d17cd"}
Sep 10 16:19:52.538963 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:52.538800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" event={"ID":"9cf14a3b-d25c-4d9a-b210-8088b003e50d","Type":"ContainerStarted","Data":"a01ad91a9ea5ee6dfd550e0182f8958302f6c03828906276381650dfc8ef2609"}
Sep 10 16:19:52.538963 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:52.538877    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:19:52.562831 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:52.562785    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" podStartSLOduration=2.184291994 podStartE2EDuration="2.562772002s" totalImagesPullingTime="378.480008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:52.561736729 +0000 UTC m=+1087.794684845" watchObservedRunningTime="2026-09-10 16:19:52.562772002 +0000 UTC m=+1087.795720103"
Sep 10 16:19:58.548823 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:19:58.548793    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:20:18.911266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.911223    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:20:18.914633 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.914608    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.917937 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.917910    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-1596ac\"/\"kube-root-ca.crt\""
Sep 10 16:20:18.919290 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.919260    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-1596ac\"/\"default-dockercfg-5rjjm\""
Sep 10 16:20:18.919433 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.919260    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-1596ac\"/\"openshift-service-ca.crt\""
Sep 10 16:20:18.919433 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.919315    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-1596ac\"/\"autores-model-1596ac-kserve-self-signed-certs\""
Sep 10 16:20:18.925128 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.925100    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:20:18.942280 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.942421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.942421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjds7\" (UniqueName: \"kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.942421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.942547 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942441    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:18.942547 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:18.942480    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.043768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.043725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.043975 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.043786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.043975 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.043936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.043983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.044047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjds7\" (UniqueName: \"kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.044113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.044152    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.044263    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.044461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.044429    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.046149 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.046122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.046492 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.046471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.053508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.053482    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjds7\" (UniqueName: \"kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7\") pod \"autores-model-1596ac-kserve-5fcd69f87-6q4pj\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") " pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.225512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.225425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:19.505893 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.505869    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:20:19.508258 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:20:19.508230    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd02ce2a_8f7d_4109_af03_fb194e19f3f5.slice/crio-cdc5bfc906598e2ecb80521ff92f52089751f0910d1544bae8d55732dd819933 WatchSource:0}: Error finding container cdc5bfc906598e2ecb80521ff92f52089751f0910d1544bae8d55732dd819933: Status 404 returned error can't find the container with id cdc5bfc906598e2ecb80521ff92f52089751f0910d1544bae8d55732dd819933
Sep 10 16:20:19.669573 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.669529    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" event={"ID":"cd02ce2a-8f7d-4109-af03-fb194e19f3f5","Type":"ContainerStarted","Data":"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"}
Sep 10 16:20:19.669573 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.669577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" event={"ID":"cd02ce2a-8f7d-4109-af03-fb194e19f3f5","Type":"ContainerStarted","Data":"cdc5bfc906598e2ecb80521ff92f52089751f0910d1544bae8d55732dd819933"}
Sep 10 16:20:19.692499 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:19.692430    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" podStartSLOduration=1.692411342 podStartE2EDuration="1.692411342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:20:19.691722243 +0000 UTC m=+1114.924670358" watchObservedRunningTime="2026-09-10 16:20:19.692411342 +0000 UTC m=+1114.925359494"
Sep 10 16:20:21.534539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.534496    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:20:21.534987 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.534692    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" podUID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" containerName="maas-api" containerID="cri-o://879372329c13451b36f72521a6dfb0c23ce2def429455f21480d47f7f19d17cd" gracePeriod=30
Sep 10 16:20:21.682862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.682812    2574 generic.go:356] "Generic (PLEG): container finished" podID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" containerID="879372329c13451b36f72521a6dfb0c23ce2def429455f21480d47f7f19d17cd" exitCode=0
Sep 10 16:20:21.683041 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.682869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" event={"ID":"9cf14a3b-d25c-4d9a-b210-8088b003e50d","Type":"ContainerDied","Data":"879372329c13451b36f72521a6dfb0c23ce2def429455f21480d47f7f19d17cd"}
Sep 10 16:20:21.792430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.792367    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:20:21.870442 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.870407    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") pod \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") "
Sep 10 16:20:21.870630 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.870450    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca\") pod \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") "
Sep 10 16:20:21.870630 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.870488    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs\") pod \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") "
Sep 10 16:20:21.870630 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.870513    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r474f\" (UniqueName: \"kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f\") pod \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\" (UID: \"9cf14a3b-d25c-4d9a-b210-8088b003e50d\") "
Sep 10 16:20:21.870914 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.870882    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9cf14a3b-d25c-4d9a-b210-8088b003e50d" (UID: "9cf14a3b-d25c-4d9a-b210-8088b003e50d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:21.872726 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.872697    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9cf14a3b-d25c-4d9a-b210-8088b003e50d" (UID: "9cf14a3b-d25c-4d9a-b210-8088b003e50d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:21.872820 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.872720    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9cf14a3b-d25c-4d9a-b210-8088b003e50d" (UID: "9cf14a3b-d25c-4d9a-b210-8088b003e50d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:21.872820 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.872766    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f" (OuterVolumeSpecName: "kube-api-access-r474f") pod "9cf14a3b-d25c-4d9a-b210-8088b003e50d" (UID: "9cf14a3b-d25c-4d9a-b210-8088b003e50d"). InnerVolumeSpecName "kube-api-access-r474f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:20:21.971356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.971302    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:21.971356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.971356    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cf14a3b-d25c-4d9a-b210-8088b003e50d-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:21.971554 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.971371    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cf14a3b-d25c-4d9a-b210-8088b003e50d-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:21.971554 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:21.971385    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r474f\" (UniqueName: \"kubernetes.io/projected/9cf14a3b-d25c-4d9a-b210-8088b003e50d-kube-api-access-r474f\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:22.688644 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:22.688616    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"
Sep 10 16:20:22.689046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:22.688650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk" event={"ID":"9cf14a3b-d25c-4d9a-b210-8088b003e50d","Type":"ContainerDied","Data":"a01ad91a9ea5ee6dfd550e0182f8958302f6c03828906276381650dfc8ef2609"}
Sep 10 16:20:22.689046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:22.688710    2574 scope.go:117] "RemoveContainer" containerID="879372329c13451b36f72521a6dfb0c23ce2def429455f21480d47f7f19d17cd"
Sep 10 16:20:22.717576 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:22.717542    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:20:22.739652 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:22.739595    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b7764c7d-68fbbd444f-q9zbk"]
Sep 10 16:20:23.266451 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:23.266411    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" path="/var/lib/kubelet/pods/9cf14a3b-d25c-4d9a-b210-8088b003e50d/volumes"
Sep 10 16:20:26.664290 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.664220    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:20:26.664792 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.664763    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" containerName="maas-api"
Sep 10 16:20:26.664792 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.664786    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" containerName="maas-api"
Sep 10 16:20:26.664894 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.664863    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cf14a3b-d25c-4d9a-b210-8088b003e50d" containerName="maas-api"
Sep 10 16:20:26.669241 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.669214    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.672553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.672528    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7a64cdb4-serving-cert\""
Sep 10 16:20:26.680339 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.680301    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:20:26.716294 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.716256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.716479 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.716373    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wddtz\" (UniqueName: \"kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.716479 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.716395    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.716479 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.716450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.817659 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.817621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.817840 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.817680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.817840 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.817705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.817840 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:20:26.817784    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-serving-cert: secret "maas-api-e2e-ait-7a64cdb4-serving-cert" not found
Sep 10 16:20:26.817986 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:20:26.817849    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls podName:17c239ee-0f0b-4bb7-9232-2ba8f710411e nodeName:}" failed. No retries permitted until 2026-09-10 16:20:27.317830883 +0000 UTC m=+1122.550778963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls") pod "maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" (UID: "17c239ee-0f0b-4bb7-9232-2ba8f710411e") : secret "maas-api-e2e-ait-7a64cdb4-serving-cert" not found
Sep 10 16:20:26.817986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.817883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wddtz\" (UniqueName: \"kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.818561 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.818534    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.820263 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.820232    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:26.842982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:26.842951    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wddtz\" (UniqueName: \"kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:27.322236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:27.322194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:27.324734 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:27.324710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") pod \"maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:27.581075 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:27.580976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:27.874765 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:27.874668    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:20:27.880987 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:20:27.880958    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c239ee_0f0b_4bb7_9232_2ba8f710411e.slice/crio-7a20ba7e81484b1ed57ec4ee8140c95cfab3660e69c70779c8ecb132f810250d WatchSource:0}: Error finding container 7a20ba7e81484b1ed57ec4ee8140c95cfab3660e69c70779c8ecb132f810250d: Status 404 returned error can't find the container with id 7a20ba7e81484b1ed57ec4ee8140c95cfab3660e69c70779c8ecb132f810250d
Sep 10 16:20:28.718965 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:28.718879    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" event={"ID":"17c239ee-0f0b-4bb7-9232-2ba8f710411e","Type":"ContainerStarted","Data":"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"}
Sep 10 16:20:28.718965 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:28.718931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" event={"ID":"17c239ee-0f0b-4bb7-9232-2ba8f710411e","Type":"ContainerStarted","Data":"7a20ba7e81484b1ed57ec4ee8140c95cfab3660e69c70779c8ecb132f810250d"}
Sep 10 16:20:28.719156 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:28.718989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:28.743350 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:28.743276    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" podStartSLOduration=2.276691209 podStartE2EDuration="2.743236543s" totalImagesPullingTime="466.545334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:20:28.741869995 +0000 UTC m=+1123.974818098" watchObservedRunningTime="2026-09-10 16:20:28.743236543 +0000 UTC m=+1123.976184645"
Sep 10 16:20:29.226546 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:29.226501    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:29.226972 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:29.226560    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:29.238804 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:29.238781    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:29.735313 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:29.735281    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:34.729768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:34.729737    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:20:44.003787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:44.003743    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:20:54.957963 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:54.957930    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:20:54.958533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:54.958126    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" podUID="bb673278-7629-4222-b91b-f14c519a5ae5" containerName="maas-api" containerID="cri-o://f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a" gracePeriod=30
Sep 10 16:20:55.221584 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.221522    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:20:55.291650 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.291610    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca\") pod \"bb673278-7629-4222-b91b-f14c519a5ae5\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") "
Sep 10 16:20:55.291829 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.291670    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") pod \"bb673278-7629-4222-b91b-f14c519a5ae5\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") "
Sep 10 16:20:55.291829 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.291729    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs\") pod \"bb673278-7629-4222-b91b-f14c519a5ae5\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") "
Sep 10 16:20:55.291829 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.291759    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjns4\" (UniqueName: \"kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4\") pod \"bb673278-7629-4222-b91b-f14c519a5ae5\" (UID: \"bb673278-7629-4222-b91b-f14c519a5ae5\") "
Sep 10 16:20:55.292087 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.292062    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb673278-7629-4222-b91b-f14c519a5ae5" (UID: "bb673278-7629-4222-b91b-f14c519a5ae5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:55.293946 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.293919    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb673278-7629-4222-b91b-f14c519a5ae5" (UID: "bb673278-7629-4222-b91b-f14c519a5ae5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:55.294061 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.293989    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4" (OuterVolumeSpecName: "kube-api-access-wjns4") pod "bb673278-7629-4222-b91b-f14c519a5ae5" (UID: "bb673278-7629-4222-b91b-f14c519a5ae5"). InnerVolumeSpecName "kube-api-access-wjns4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:20:55.294101 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.294088    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bb673278-7629-4222-b91b-f14c519a5ae5" (UID: "bb673278-7629-4222-b91b-f14c519a5ae5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:55.392636 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.392605    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:55.392636 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.392634    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb673278-7629-4222-b91b-f14c519a5ae5-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:55.392822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.392643    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjns4\" (UniqueName: \"kubernetes.io/projected/bb673278-7629-4222-b91b-f14c519a5ae5-kube-api-access-wjns4\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:55.392822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.392654    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb673278-7629-4222-b91b-f14c519a5ae5-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:55.849118 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.849081    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb673278-7629-4222-b91b-f14c519a5ae5" containerID="f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a" exitCode=0
Sep 10 16:20:55.849293 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.849122    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"
Sep 10 16:20:55.849293 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.849145    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" event={"ID":"bb673278-7629-4222-b91b-f14c519a5ae5","Type":"ContainerDied","Data":"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"}
Sep 10 16:20:55.849293 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.849196    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj" event={"ID":"bb673278-7629-4222-b91b-f14c519a5ae5","Type":"ContainerDied","Data":"d679b1dcf554f1a5993efe901a717d7d900a80ec3af257ef2692385d2569785b"}
Sep 10 16:20:55.849293 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.849219    2574 scope.go:117] "RemoveContainer" containerID="f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"
Sep 10 16:20:55.858971 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.858951    2574 scope.go:117] "RemoveContainer" containerID="f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"
Sep 10 16:20:55.859263 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:20:55.859245    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a\": container with ID starting with f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a not found: ID does not exist" containerID="f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"
Sep 10 16:20:55.859308 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.859272    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a"} err="failed to get container status \"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a\": rpc error: code = NotFound desc = could not find container \"f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a\": container with ID starting with f4e872b44b9ea05c925606031096f9d935c10763d3bcedd3db407f049359330a not found: ID does not exist"
Sep 10 16:20:55.883671 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.883453    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:20:55.886772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:55.886748    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-1596ac-746f647786-765mj"]
Sep 10 16:20:57.267358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:57.267307    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb673278-7629-4222-b91b-f14c519a5ae5" path="/var/lib/kubelet/pods/bb673278-7629-4222-b91b-f14c519a5ae5/volumes"
Sep 10 16:20:58.502035 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:58.501994    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:20:59.023702 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.023659    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" podUID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" containerName="main" containerID="cri-o://96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07" gracePeriod=45
Sep 10 16:20:59.274027 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.273967    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:59.323753 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.323713    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjds7\" (UniqueName: \"kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.323926 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.323836    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.323926 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.323911    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.324042 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.323998    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.324101 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324052    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.324153 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324101    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home\") pod \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\" (UID: \"cd02ce2a-8f7d-4109-af03-fb194e19f3f5\") "
Sep 10 16:20:59.324238 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:20:59.324514 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324473    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.324514 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324477    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home" (OuterVolumeSpecName: "home") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:20:59.324697 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.324610    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:20:59.326033 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.326007    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm" (OuterVolumeSpecName: "dshm") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:20:59.326161 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.326071    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7" (OuterVolumeSpecName: "kube-api-access-fjds7") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "kube-api-access-fjds7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:20:59.326229 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.326208    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd02ce2a-8f7d-4109-af03-fb194e19f3f5" (UID: "cd02ce2a-8f7d-4109-af03-fb194e19f3f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:59.425120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.425084    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.425120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.425117    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.425120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.425127    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjds7\" (UniqueName: \"kubernetes.io/projected/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-kube-api-access-fjds7\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.425382 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.425135    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.425382 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.425144    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd02ce2a-8f7d-4109-af03-fb194e19f3f5-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:20:59.869508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.869469    2574 generic.go:356] "Generic (PLEG): container finished" podID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" containerID="96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07" exitCode=0
Sep 10 16:20:59.869984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.869516    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"
Sep 10 16:20:59.869984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.869563    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" event={"ID":"cd02ce2a-8f7d-4109-af03-fb194e19f3f5","Type":"ContainerDied","Data":"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"}
Sep 10 16:20:59.869984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.869615    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj" event={"ID":"cd02ce2a-8f7d-4109-af03-fb194e19f3f5","Type":"ContainerDied","Data":"cdc5bfc906598e2ecb80521ff92f52089751f0910d1544bae8d55732dd819933"}
Sep 10 16:20:59.869984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.869644    2574 scope.go:117] "RemoveContainer" containerID="96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"
Sep 10 16:20:59.878933 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.878914    2574 scope.go:117] "RemoveContainer" containerID="96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"
Sep 10 16:20:59.879204 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:20:59.879186    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07\": container with ID starting with 96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07 not found: ID does not exist" containerID="96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"
Sep 10 16:20:59.879241 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.879214    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07"} err="failed to get container status \"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07\": rpc error: code = NotFound desc = could not find container \"96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07\": container with ID starting with 96e729eedc8d81d24e3d71a9cdabaeae98da30574f59a93750598e229a4fcb07 not found: ID does not exist"
Sep 10 16:20:59.897310 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.897286    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:20:59.900719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:20:59.900692    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-1596ac/autores-model-1596ac-kserve-5fcd69f87-6q4pj"]
Sep 10 16:21:01.266272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:01.266234    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" path="/var/lib/kubelet/pods/cd02ce2a-8f7d-4109-af03-fb194e19f3f5/volumes"
Sep 10 16:21:07.605169 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.605133    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:21:07.605569 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.605362    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" podUID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" containerName="maas-api" containerID="cri-o://a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a" gracePeriod=30
Sep 10 16:21:07.860988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.860920    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:21:07.904727 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.904697    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") pod \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") "
Sep 10 16:21:07.904891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.904808    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs\") pod \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") "
Sep 10 16:21:07.904891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.904834    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wddtz\" (UniqueName: \"kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz\") pod \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") "
Sep 10 16:21:07.904891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.904862    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca\") pod \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\" (UID: \"17c239ee-0f0b-4bb7-9232-2ba8f710411e\") "
Sep 10 16:21:07.905288 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.905253    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17c239ee-0f0b-4bb7-9232-2ba8f710411e" (UID: "17c239ee-0f0b-4bb7-9232-2ba8f710411e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:21:07.907055 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.907025    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17c239ee-0f0b-4bb7-9232-2ba8f710411e" (UID: "17c239ee-0f0b-4bb7-9232-2ba8f710411e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:21:07.907187 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.907071    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "17c239ee-0f0b-4bb7-9232-2ba8f710411e" (UID: "17c239ee-0f0b-4bb7-9232-2ba8f710411e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:21:07.907362 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.907319    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz" (OuterVolumeSpecName: "kube-api-access-wddtz") pod "17c239ee-0f0b-4bb7-9232-2ba8f710411e" (UID: "17c239ee-0f0b-4bb7-9232-2ba8f710411e"). InnerVolumeSpecName "kube-api-access-wddtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:21:07.909781 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.909753    2574 generic.go:356] "Generic (PLEG): container finished" podID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" containerID="a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a" exitCode=0
Sep 10 16:21:07.909861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.909794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" event={"ID":"17c239ee-0f0b-4bb7-9232-2ba8f710411e","Type":"ContainerDied","Data":"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"}
Sep 10 16:21:07.909861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.909836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds" event={"ID":"17c239ee-0f0b-4bb7-9232-2ba8f710411e","Type":"ContainerDied","Data":"7a20ba7e81484b1ed57ec4ee8140c95cfab3660e69c70779c8ecb132f810250d"}
Sep 10 16:21:07.909861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.909857    2574 scope.go:117] "RemoveContainer" containerID="a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"
Sep 10 16:21:07.909983 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.909799    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"
Sep 10 16:21:07.926441 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.926292    2574 scope.go:117] "RemoveContainer" containerID="a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"
Sep 10 16:21:07.927017 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:07.926981    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a\": container with ID starting with a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a not found: ID does not exist" containerID="a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"
Sep 10 16:21:07.927017 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.927018    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a"} err="failed to get container status \"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a\": rpc error: code = NotFound desc = could not find container \"a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a\": container with ID starting with a349d9264f283c74d16003ee974d1472814e538c7421f5a2596a1c3d3a126d5a not found: ID does not exist"
Sep 10 16:21:07.972622 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.972589    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:21:07.993707 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:07.993668    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a64cdb4-76cb8b79cc-m9fds"]
Sep 10 16:21:08.005625 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:08.005597    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:08.005625 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:08.005624    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wddtz\" (UniqueName: \"kubernetes.io/projected/17c239ee-0f0b-4bb7-9232-2ba8f710411e-kube-api-access-wddtz\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:08.005822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:08.005635    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17c239ee-0f0b-4bb7-9232-2ba8f710411e-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:08.005822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:08.005647    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17c239ee-0f0b-4bb7-9232-2ba8f710411e-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:09.265725 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:09.265687    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" path="/var/lib/kubelet/pods/17c239ee-0f0b-4bb7-9232-2ba8f710411e/volumes"
Sep 10 16:21:13.007511 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007472    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007835    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" containerName="maas-api"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007847    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" containerName="maas-api"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007856    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" containerName="main"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007862    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" containerName="main"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007870    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb673278-7629-4222-b91b-f14c519a5ae5" containerName="maas-api"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007876    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb673278-7629-4222-b91b-f14c519a5ae5" containerName="maas-api"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007928    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd02ce2a-8f7d-4109-af03-fb194e19f3f5" containerName="main"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007937    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb673278-7629-4222-b91b-f14c519a5ae5" containerName="maas-api"
Sep 10 16:21:13.007956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.007943    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="17c239ee-0f0b-4bb7-9232-2ba8f710411e" containerName="maas-api"
Sep 10 16:21:13.012109 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.012085    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.015250 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.015225    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e22c72\"/\"openshift-service-ca.crt\""
Sep 10 16:21:13.016358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.016311    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e22c72\"/\"kube-root-ca.crt\""
Sep 10 16:21:13.016637 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.016616    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e22c72\"/\"default-dockercfg-fz76l\""
Sep 10 16:21:13.016701 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.016630    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e22c72\"/\"test-model-e22c72-kserve-self-signed-certs\""
Sep 10 16:21:13.029385 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.029356    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:21:13.151190 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.151190 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkfvx\" (UniqueName: \"kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.151491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.151491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.151491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.151491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.151471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252319 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252499 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252499 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252483    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252617 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkfvx\" (UniqueName: \"kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252617 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.252709 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.253107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252829    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.253107 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.252891    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.253431 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.253406    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.255215 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.255183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.255440 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.255424    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.265697 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.265636    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkfvx\" (UniqueName: \"kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx\") pod \"test-model-e22c72-kserve-5d9c877d8c-h79gl\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") " pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.321257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.321209    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:13.522647 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.522561    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" podUID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" containerName="main" containerID="cri-o://d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5" gracePeriod=45
Sep 10 16:21:13.600414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.600386    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:21:13.602424 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:21:13.602394    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72f74003_6513_42e3_826b_8b28d1dbbe2e.slice/crio-befb2b74f3a2c08f54a45a011dc71f853c8df7676b243b5312a60ff923fd4386 WatchSource:0}: Error finding container befb2b74f3a2c08f54a45a011dc71f853c8df7676b243b5312a60ff923fd4386: Status 404 returned error can't find the container with id befb2b74f3a2c08f54a45a011dc71f853c8df7676b243b5312a60ff923fd4386
Sep 10 16:21:13.758758 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.758726    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:21:13.858614 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858579    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.858614 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858619    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.858935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858655    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp6zp\" (UniqueName: \"kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.858935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858796    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.858935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858877    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home" (OuterVolumeSpecName: "home") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:13.858935 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858887    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.859121 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858993    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm\") pod \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\" (UID: \"50cbb9a9-42ac-4274-8d20-6227e4999b4c\") "
Sep 10 16:21:13.859121 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.858984    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:13.859216 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.859111    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:13.859314 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.859297    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:13.859393 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.859319    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:13.859393 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.859360    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:13.861007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.860988    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm" (OuterVolumeSpecName: "dshm") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:13.861276 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.861255    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:21:13.861345 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.861274    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp" (OuterVolumeSpecName: "kube-api-access-mp6zp") pod "50cbb9a9-42ac-4274-8d20-6227e4999b4c" (UID: "50cbb9a9-42ac-4274-8d20-6227e4999b4c"). InnerVolumeSpecName "kube-api-access-mp6zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:21:13.942501 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.942463    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" event={"ID":"72f74003-6513-42e3-826b-8b28d1dbbe2e","Type":"ContainerStarted","Data":"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"}
Sep 10 16:21:13.942722 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.942521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" event={"ID":"72f74003-6513-42e3-826b-8b28d1dbbe2e","Type":"ContainerStarted","Data":"befb2b74f3a2c08f54a45a011dc71f853c8df7676b243b5312a60ff923fd4386"}
Sep 10 16:21:13.943942 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.943907    2574 generic.go:356] "Generic (PLEG): container finished" podID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" containerID="d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5" exitCode=0
Sep 10 16:21:13.944081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.943946    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"
Sep 10 16:21:13.944081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.943987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" event={"ID":"50cbb9a9-42ac-4274-8d20-6227e4999b4c","Type":"ContainerDied","Data":"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"}
Sep 10 16:21:13.944081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.944025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg" event={"ID":"50cbb9a9-42ac-4274-8d20-6227e4999b4c","Type":"ContainerDied","Data":"51373b87b55b9a2dc560135b10c574df60889988363c3d66239650aff00b87e4"}
Sep 10 16:21:13.944081 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.944041    2574 scope.go:117] "RemoveContainer" containerID="d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"
Sep 10 16:21:13.953644 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.953628    2574 scope.go:117] "RemoveContainer" containerID="d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"
Sep 10 16:21:13.953979 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:13.953944    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5\": container with ID starting with d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5 not found: ID does not exist" containerID="d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"
Sep 10 16:21:13.954045 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.953992    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5"} err="failed to get container status \"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5\": rpc error: code = NotFound desc = could not find container \"d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5\": container with ID starting with d7b67f3e48b7d3010b63aff226b70a21cfdb08f821153f992a7b6de5e59891e5 not found: ID does not exist"
Sep 10 16:21:13.960476 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.960455    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50cbb9a9-42ac-4274-8d20-6227e4999b4c-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:13.960554 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.960478    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp6zp\" (UniqueName: \"kubernetes.io/projected/50cbb9a9-42ac-4274-8d20-6227e4999b4c-kube-api-access-mp6zp\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:13.960554 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:13.960492    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50cbb9a9-42ac-4274-8d20-6227e4999b4c-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:14.007788 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.007732    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" podStartSLOduration=2.007717099 podStartE2EDuration="2.007717099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:14.005779062 +0000 UTC m=+1169.238727166" watchObservedRunningTime="2026-09-10 16:21:14.007717099 +0000 UTC m=+1169.240665200"
Sep 10 16:21:14.037283 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.037244    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:21:14.039615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.039594    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-aa8ef0bd-kserve-59b99664f8-fmcmg"]
Sep 10 16:21:14.092407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.092373    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:14.092743 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.092730    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" containerName="main"
Sep 10 16:21:14.092786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.092745    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" containerName="main"
Sep 10 16:21:14.092819 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.092808    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" containerName="main"
Sep 10 16:21:14.097145 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.097089    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.100685 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.100658    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0e28e849-serving-cert\""
Sep 10 16:21:14.111921 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.111853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:14.162756 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.162717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.162938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.162773    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.162938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.162800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.162938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.162866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k79dd\" (UniqueName: \"kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.264234 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.264196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.264401 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.264270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.264401 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.264299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.264401 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:14.264323    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-serving-cert: secret "maas-api-e2e-derive-0e28e849-serving-cert" not found
Sep 10 16:21:14.264401 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.264389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k79dd\" (UniqueName: \"kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.264620 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:14.264412    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls podName:5c400b94-f991-4931-a641-602de99c4f2e nodeName:}" failed. No retries permitted until 2026-09-10 16:21:14.764392689 +0000 UTC m=+1169.997340770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls") pod "maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" (UID: "5c400b94-f991-4931-a641-602de99c4f2e") : secret "maas-api-e2e-derive-0e28e849-serving-cert" not found
Sep 10 16:21:14.265054 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.265031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.266985 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.266966    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.274695 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.274669    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k79dd\" (UniqueName: \"kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.768357 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.768288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:14.770624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:14.770604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") pod \"maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:15.009884 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.009836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:15.267609 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.267571    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cbb9a9-42ac-4274-8d20-6227e4999b4c" path="/var/lib/kubelet/pods/50cbb9a9-42ac-4274-8d20-6227e4999b4c/volumes"
Sep 10 16:21:15.284927 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.284900    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:15.287344 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:21:15.287296    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c400b94_f991_4931_a641_602de99c4f2e.slice/crio-e97e045a71a43f0890acc9ef940480551d55ed336f1b3f2be5de48255b5f2d20 WatchSource:0}: Error finding container e97e045a71a43f0890acc9ef940480551d55ed336f1b3f2be5de48255b5f2d20: Status 404 returned error can't find the container with id e97e045a71a43f0890acc9ef940480551d55ed336f1b3f2be5de48255b5f2d20
Sep 10 16:21:15.957012 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.956977    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" event={"ID":"5c400b94-f991-4931-a641-602de99c4f2e","Type":"ContainerStarted","Data":"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"}
Sep 10 16:21:15.957179 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.957031    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" event={"ID":"5c400b94-f991-4931-a641-602de99c4f2e","Type":"ContainerStarted","Data":"e97e045a71a43f0890acc9ef940480551d55ed336f1b3f2be5de48255b5f2d20"}
Sep 10 16:21:15.957179 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.957110    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:15.986380 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:15.986299    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" podStartSLOduration=1.4414421499999999 podStartE2EDuration="1.986282364s" totalImagesPullingTime="544.840214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:15.982577425 +0000 UTC m=+1171.215525527" watchObservedRunningTime="2026-09-10 16:21:15.986282364 +0000 UTC m=+1171.219230465"
Sep 10 16:21:20.193633 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.193591    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:21:20.197180 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.197154    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.203827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.203789    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-555853-serving-cert\""
Sep 10 16:21:20.211337 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.211291    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:21:20.319849 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.319809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.320031 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.319864    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjsc\" (UniqueName: \"kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.320031 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.319885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.320031 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.319947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.421027 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.420985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.421206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.421049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjsc\" (UniqueName: \"kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.421206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.421076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.421206 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.421113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.421311 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:20.421289    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-555853-serving-cert: secret "maas-api-e2e-explicit-555853-serving-cert" not found
Sep 10 16:21:20.421384 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:20.421370    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls podName:969aea64-77a1-441e-9d35-ea846c16cb3b nodeName:}" failed. No retries permitted until 2026-09-10 16:21:20.921349822 +0000 UTC m=+1176.154297916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls") pod "maas-api-e2e-explicit-555853-cf44bd96f-gzprd" (UID: "969aea64-77a1-441e-9d35-ea846c16cb3b") : secret "maas-api-e2e-explicit-555853-serving-cert" not found
Sep 10 16:21:20.421710 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.421680    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.423531 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.423513    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.433814 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.433785    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjsc\" (UniqueName: \"kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.927438 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.927398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:20.929803 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:20.929783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") pod \"maas-api-e2e-explicit-555853-cf44bd96f-gzprd\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:21.110026 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:21.109989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:21.390806 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:21.390772    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:21:21.394560 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:21:21.394527    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969aea64_77a1_441e_9d35_ea846c16cb3b.slice/crio-611fb7e699663a3493c4adea9bb611e42af26f7ff1b776fd4553ad60448401b6 WatchSource:0}: Error finding container 611fb7e699663a3493c4adea9bb611e42af26f7ff1b776fd4553ad60448401b6: Status 404 returned error can't find the container with id 611fb7e699663a3493c4adea9bb611e42af26f7ff1b776fd4553ad60448401b6
Sep 10 16:21:21.967544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:21.967513    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:21.986301 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:21.986267    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" event={"ID":"969aea64-77a1-441e-9d35-ea846c16cb3b","Type":"ContainerStarted","Data":"611fb7e699663a3493c4adea9bb611e42af26f7ff1b776fd4553ad60448401b6"}
Sep 10 16:21:22.992956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:22.992919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" event={"ID":"969aea64-77a1-441e-9d35-ea846c16cb3b","Type":"ContainerStarted","Data":"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"}
Sep 10 16:21:22.993390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:22.992986    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:23.015306 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:23.015246    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" podStartSLOduration=2.399851268 podStartE2EDuration="3.015228902s" totalImagesPullingTime="615.377634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:23.013671042 +0000 UTC m=+1178.246619145" watchObservedRunningTime="2026-09-10 16:21:23.015228902 +0000 UTC m=+1178.248177004"
Sep 10 16:21:23.321847 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:23.321804    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:23.322063 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:23.321863    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:23.334832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:23.334801    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:24.009488 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:24.009458    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:21:29.003412 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:29.003376    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:21:44.718668 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:44.718628    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:44.719187 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:44.718860    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" podUID="5c400b94-f991-4931-a641-602de99c4f2e" containerName="maas-api" containerID="cri-o://a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0" gracePeriod=30
Sep 10 16:21:44.964912 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:44.964887    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:45.049796 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.049758    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs\") pod \"5c400b94-f991-4931-a641-602de99c4f2e\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") "
Sep 10 16:21:45.049796 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.049802    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca\") pod \"5c400b94-f991-4931-a641-602de99c4f2e\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") "
Sep 10 16:21:45.050020 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.049865    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") pod \"5c400b94-f991-4931-a641-602de99c4f2e\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") "
Sep 10 16:21:45.050020 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.049922    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k79dd\" (UniqueName: \"kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd\") pod \"5c400b94-f991-4931-a641-602de99c4f2e\" (UID: \"5c400b94-f991-4931-a641-602de99c4f2e\") "
Sep 10 16:21:45.050240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.050206    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5c400b94-f991-4931-a641-602de99c4f2e" (UID: "5c400b94-f991-4931-a641-602de99c4f2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:21:45.051961 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.051932    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5c400b94-f991-4931-a641-602de99c4f2e" (UID: "5c400b94-f991-4931-a641-602de99c4f2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:21:45.052083 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.051999    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5c400b94-f991-4931-a641-602de99c4f2e" (UID: "5c400b94-f991-4931-a641-602de99c4f2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:21:45.052083 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.052070    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd" (OuterVolumeSpecName: "kube-api-access-k79dd") pod "5c400b94-f991-4931-a641-602de99c4f2e" (UID: "5c400b94-f991-4931-a641-602de99c4f2e"). InnerVolumeSpecName "kube-api-access-k79dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:21:45.109112 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.109079    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c400b94-f991-4931-a641-602de99c4f2e" containerID="a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0" exitCode=0
Sep 10 16:21:45.109300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.109154    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" event={"ID":"5c400b94-f991-4931-a641-602de99c4f2e","Type":"ContainerDied","Data":"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"}
Sep 10 16:21:45.109300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.109192    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"
Sep 10 16:21:45.109300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.109238    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm" event={"ID":"5c400b94-f991-4931-a641-602de99c4f2e","Type":"ContainerDied","Data":"e97e045a71a43f0890acc9ef940480551d55ed336f1b3f2be5de48255b5f2d20"}
Sep 10 16:21:45.109300 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.109274    2574 scope.go:117] "RemoveContainer" containerID="a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"
Sep 10 16:21:45.123454 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.123425    2574 scope.go:117] "RemoveContainer" containerID="a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"
Sep 10 16:21:45.123803 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:21:45.123770    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0\": container with ID starting with a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0 not found: ID does not exist" containerID="a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"
Sep 10 16:21:45.123900 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.123807    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0"} err="failed to get container status \"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0\": rpc error: code = NotFound desc = could not find container \"a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0\": container with ID starting with a40f857141e3b45eb6b63409afdad0030e9b1c6c58e53c98de04ec480cb34ce0 not found: ID does not exist"
Sep 10 16:21:45.144285 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.144261    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:45.147630 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.147606    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0e28e849-69c6dcc466-vgscm"]
Sep 10 16:21:45.151475 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.151456    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k79dd\" (UniqueName: \"kubernetes.io/projected/5c400b94-f991-4931-a641-602de99c4f2e-kube-api-access-k79dd\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:45.151533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.151479    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:45.151533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.151490    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c400b94-f991-4931-a641-602de99c4f2e-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:45.151533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.151499    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5c400b94-f991-4931-a641-602de99c4f2e-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:21:45.255809 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.255780    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:21:45.256373 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.256353    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:21:45.266652 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:21:45.266625    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c400b94-f991-4931-a641-602de99c4f2e" path="/var/lib/kubelet/pods/5c400b94-f991-4931-a641-602de99c4f2e/volumes"
Sep 10 16:22:30.328702 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:30.328668    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:22:38.258279 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.258198    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:22:38.258839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.258493    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" podUID="969aea64-77a1-441e-9d35-ea846c16cb3b" containerName="maas-api" containerID="cri-o://a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376" gracePeriod=30
Sep 10 16:22:38.507025 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.506999    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:22:38.635069 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.635025    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs\") pod \"969aea64-77a1-441e-9d35-ea846c16cb3b\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") "
Sep 10 16:22:38.635288 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.635099    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fjsc\" (UniqueName: \"kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc\") pod \"969aea64-77a1-441e-9d35-ea846c16cb3b\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") "
Sep 10 16:22:38.635288 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.635155    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") pod \"969aea64-77a1-441e-9d35-ea846c16cb3b\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") "
Sep 10 16:22:38.635288 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.635182    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca\") pod \"969aea64-77a1-441e-9d35-ea846c16cb3b\" (UID: \"969aea64-77a1-441e-9d35-ea846c16cb3b\") "
Sep 10 16:22:38.635682 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.635649    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "969aea64-77a1-441e-9d35-ea846c16cb3b" (UID: "969aea64-77a1-441e-9d35-ea846c16cb3b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:22:38.637308 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.637278    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "969aea64-77a1-441e-9d35-ea846c16cb3b" (UID: "969aea64-77a1-441e-9d35-ea846c16cb3b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:38.637425 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.637297    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "969aea64-77a1-441e-9d35-ea846c16cb3b" (UID: "969aea64-77a1-441e-9d35-ea846c16cb3b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:38.637425 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.637317    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc" (OuterVolumeSpecName: "kube-api-access-5fjsc") pod "969aea64-77a1-441e-9d35-ea846c16cb3b" (UID: "969aea64-77a1-441e-9d35-ea846c16cb3b"). InnerVolumeSpecName "kube-api-access-5fjsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:38.736694 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.736652    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:38.736694 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.736687    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969aea64-77a1-441e-9d35-ea846c16cb3b-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:38.736694 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.736699    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969aea64-77a1-441e-9d35-ea846c16cb3b-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:38.736929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:38.736707    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fjsc\" (UniqueName: \"kubernetes.io/projected/969aea64-77a1-441e-9d35-ea846c16cb3b-kube-api-access-5fjsc\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:39.361979 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.361943    2574 generic.go:356] "Generic (PLEG): container finished" podID="969aea64-77a1-441e-9d35-ea846c16cb3b" containerID="a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376" exitCode=0
Sep 10 16:22:39.362395 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.361985    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"
Sep 10 16:22:39.362395 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.362002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" event={"ID":"969aea64-77a1-441e-9d35-ea846c16cb3b","Type":"ContainerDied","Data":"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"}
Sep 10 16:22:39.362395 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.362050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd" event={"ID":"969aea64-77a1-441e-9d35-ea846c16cb3b","Type":"ContainerDied","Data":"611fb7e699663a3493c4adea9bb611e42af26f7ff1b776fd4553ad60448401b6"}
Sep 10 16:22:39.362395 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.362072    2574 scope.go:117] "RemoveContainer" containerID="a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"
Sep 10 16:22:39.370796 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.370775    2574 scope.go:117] "RemoveContainer" containerID="a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"
Sep 10 16:22:39.371073 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:22:39.371054    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376\": container with ID starting with a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376 not found: ID does not exist" containerID="a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"
Sep 10 16:22:39.371119 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.371080    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376"} err="failed to get container status \"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376\": rpc error: code = NotFound desc = could not find container \"a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376\": container with ID starting with a35a90ec12dbba03d23033f1e4e9dd10c4ee39bef52f43fa9fce9da42de55376 not found: ID does not exist"
Sep 10 16:22:39.384298 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.384261    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:22:39.393702 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:39.393671    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-555853-cf44bd96f-gzprd"]
Sep 10 16:22:41.268402 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.268367    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969aea64-77a1-441e-9d35-ea846c16cb3b" path="/var/lib/kubelet/pods/969aea64-77a1-441e-9d35-ea846c16cb3b/volumes"
Sep 10 16:22:41.292032 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.291994    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:22:41.292321 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.292271    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" podUID="aae45a17-c19f-4319-b917-48385e06c6a3" containerName="maas-api" containerID="cri-o://f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d" gracePeriod=30
Sep 10 16:22:41.538982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.538957    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:22:41.663266 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.663227    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca\") pod \"aae45a17-c19f-4319-b917-48385e06c6a3\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") "
Sep 10 16:22:41.663497 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.663282    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls\") pod \"aae45a17-c19f-4319-b917-48385e06c6a3\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") "
Sep 10 16:22:41.663497 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.663367    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl7mv\" (UniqueName: \"kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv\") pod \"aae45a17-c19f-4319-b917-48385e06c6a3\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") "
Sep 10 16:22:41.663497 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.663421    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs\") pod \"aae45a17-c19f-4319-b917-48385e06c6a3\" (UID: \"aae45a17-c19f-4319-b917-48385e06c6a3\") "
Sep 10 16:22:41.664019 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.663987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aae45a17-c19f-4319-b917-48385e06c6a3" (UID: "aae45a17-c19f-4319-b917-48385e06c6a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:22:41.665762 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.665731    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aae45a17-c19f-4319-b917-48385e06c6a3" (UID: "aae45a17-c19f-4319-b917-48385e06c6a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:41.665879 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.665842    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aae45a17-c19f-4319-b917-48385e06c6a3" (UID: "aae45a17-c19f-4319-b917-48385e06c6a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:41.665879 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.665857    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv" (OuterVolumeSpecName: "kube-api-access-fl7mv") pod "aae45a17-c19f-4319-b917-48385e06c6a3" (UID: "aae45a17-c19f-4319-b917-48385e06c6a3"). InnerVolumeSpecName "kube-api-access-fl7mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:41.764394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.764323    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl7mv\" (UniqueName: \"kubernetes.io/projected/aae45a17-c19f-4319-b917-48385e06c6a3-kube-api-access-fl7mv\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:41.764394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.764388    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:41.764394 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.764397    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae45a17-c19f-4319-b917-48385e06c6a3-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:41.764642 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:41.764408    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aae45a17-c19f-4319-b917-48385e06c6a3-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:42.379048 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.379011    2574 generic.go:356] "Generic (PLEG): container finished" podID="aae45a17-c19f-4319-b917-48385e06c6a3" containerID="f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d" exitCode=0
Sep 10 16:22:42.379577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.379064    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"
Sep 10 16:22:42.379577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.379065    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" event={"ID":"aae45a17-c19f-4319-b917-48385e06c6a3","Type":"ContainerDied","Data":"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"}
Sep 10 16:22:42.379577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.379203    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4" event={"ID":"aae45a17-c19f-4319-b917-48385e06c6a3","Type":"ContainerDied","Data":"a6cee2602596d81e9dc97c300ce8c97936adb63aecaafd7bd7fe79cb4b001b59"}
Sep 10 16:22:42.379577 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.379230    2574 scope.go:117] "RemoveContainer" containerID="f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"
Sep 10 16:22:42.388294 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.388270    2574 scope.go:117] "RemoveContainer" containerID="f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"
Sep 10 16:22:42.388583 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:22:42.388565    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d\": container with ID starting with f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d not found: ID does not exist" containerID="f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"
Sep 10 16:22:42.388629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.388593    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d"} err="failed to get container status \"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d\": rpc error: code = NotFound desc = could not find container \"f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d\": container with ID starting with f61dc8e6d8a5de55148b7f521af194bf49e5268ace6c188fd3aea893be12186d not found: ID does not exist"
Sep 10 16:22:42.406833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.406803    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:22:42.413129 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:42.413102    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-37c805-86bc84ccfc-l5vz4"]
Sep 10 16:22:43.266897 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:43.266860    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aae45a17-c19f-4319-b917-48385e06c6a3" path="/var/lib/kubelet/pods/aae45a17-c19f-4319-b917-48385e06c6a3/volumes"
Sep 10 16:22:45.347801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.347764    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" podUID="72f74003-6513-42e3-826b-8b28d1dbbe2e" containerName="main" containerID="cri-o://e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd" gracePeriod=45
Sep 10 16:22:45.605599 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.605574    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:22:45.699968 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.699928    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.699968 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.699975    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.700214 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700040    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.700214 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700085    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.700214 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700117    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.700214 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700168    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkfvx\" (UniqueName: \"kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx\") pod \"72f74003-6513-42e3-826b-8b28d1dbbe2e\" (UID: \"72f74003-6513-42e3-826b-8b28d1dbbe2e\") "
Sep 10 16:22:45.700458 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700246    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home" (OuterVolumeSpecName: "home") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:45.700458 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700364    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache" (OuterVolumeSpecName: "model-cache") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:45.700458 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700431    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:45.700587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700482    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:45.700587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.700500    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:45.702488 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.702456    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm" (OuterVolumeSpecName: "dshm") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:45.702488 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.702473    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:45.702641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.702548    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx" (OuterVolumeSpecName: "kube-api-access-wkfvx") pod "72f74003-6513-42e3-826b-8b28d1dbbe2e" (UID: "72f74003-6513-42e3-826b-8b28d1dbbe2e"). InnerVolumeSpecName "kube-api-access-wkfvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:45.801719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.801679    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:45.801719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.801712    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72f74003-6513-42e3-826b-8b28d1dbbe2e-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:45.801719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.801724    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkfvx\" (UniqueName: \"kubernetes.io/projected/72f74003-6513-42e3-826b-8b28d1dbbe2e-kube-api-access-wkfvx\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:45.801942 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:45.801734    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72f74003-6513-42e3-826b-8b28d1dbbe2e-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:22:46.399821 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.399785    2574 generic.go:356] "Generic (PLEG): container finished" podID="72f74003-6513-42e3-826b-8b28d1dbbe2e" containerID="e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd" exitCode=0
Sep 10 16:22:46.400282 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.399841    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"
Sep 10 16:22:46.400282 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.399866    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" event={"ID":"72f74003-6513-42e3-826b-8b28d1dbbe2e","Type":"ContainerDied","Data":"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"}
Sep 10 16:22:46.400282 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.399909    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl" event={"ID":"72f74003-6513-42e3-826b-8b28d1dbbe2e","Type":"ContainerDied","Data":"befb2b74f3a2c08f54a45a011dc71f853c8df7676b243b5312a60ff923fd4386"}
Sep 10 16:22:46.400282 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.399926    2574 scope.go:117] "RemoveContainer" containerID="e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"
Sep 10 16:22:46.409538 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.409516    2574 scope.go:117] "RemoveContainer" containerID="e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"
Sep 10 16:22:46.409817 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:22:46.409799    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd\": container with ID starting with e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd not found: ID does not exist" containerID="e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"
Sep 10 16:22:46.409867 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.409828    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd"} err="failed to get container status \"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd\": rpc error: code = NotFound desc = could not find container \"e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd\": container with ID starting with e3ab4888fded568fe1580ea9a49cd06fedb2dd86553dfdd6b53b09a86521cbdd not found: ID does not exist"
Sep 10 16:22:46.423872 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.423841    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:22:46.428473 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:46.428448    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e22c72/test-model-e22c72-kserve-5d9c877d8c-h79gl"]
Sep 10 16:22:47.266231 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:22:47.266190    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f74003-6513-42e3-826b-8b28d1dbbe2e" path="/var/lib/kubelet/pods/72f74003-6513-42e3-826b-8b28d1dbbe2e/volumes"
Sep 10 16:23:01.045054 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.045017    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:23:01.045497 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.045232    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" podUID="db85115f-cc59-4931-b5f3-43b84588c43e" containerName="maas-api" containerID="cri-o://ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672" gracePeriod=30
Sep 10 16:23:01.300555 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.300488    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:23:01.443352 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.443288    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qc52\" (UniqueName: \"kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52\") pod \"db85115f-cc59-4931-b5f3-43b84588c43e\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") "
Sep 10 16:23:01.443597 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.443370    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs\") pod \"db85115f-cc59-4931-b5f3-43b84588c43e\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") "
Sep 10 16:23:01.443597 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.443405    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") pod \"db85115f-cc59-4931-b5f3-43b84588c43e\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") "
Sep 10 16:23:01.443597 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.443429    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca\") pod \"db85115f-cc59-4931-b5f3-43b84588c43e\" (UID: \"db85115f-cc59-4931-b5f3-43b84588c43e\") "
Sep 10 16:23:01.444017 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.443888    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db85115f-cc59-4931-b5f3-43b84588c43e" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:23:01.445538 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.445514    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52" (OuterVolumeSpecName: "kube-api-access-5qc52") pod "db85115f-cc59-4931-b5f3-43b84588c43e" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e"). InnerVolumeSpecName "kube-api-access-5qc52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:01.445933 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.445909    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db85115f-cc59-4931-b5f3-43b84588c43e" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:23:01.445982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.445933    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "db85115f-cc59-4931-b5f3-43b84588c43e" (UID: "db85115f-cc59-4931-b5f3-43b84588c43e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:23:01.478390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.478357    2574 generic.go:356] "Generic (PLEG): container finished" podID="db85115f-cc59-4931-b5f3-43b84588c43e" containerID="ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672" exitCode=0
Sep 10 16:23:01.478528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.478395    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"
Sep 10 16:23:01.478528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.478412    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" event={"ID":"db85115f-cc59-4931-b5f3-43b84588c43e","Type":"ContainerDied","Data":"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"}
Sep 10 16:23:01.478528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.478464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x" event={"ID":"db85115f-cc59-4931-b5f3-43b84588c43e","Type":"ContainerDied","Data":"eaf64903003b4eb6570dc54a6fab143a6a2ed000d22129565e51538830882668"}
Sep 10 16:23:01.478528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.478486    2574 scope.go:117] "RemoveContainer" containerID="ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"
Sep 10 16:23:01.488257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.488238    2574 scope.go:117] "RemoveContainer" containerID="ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"
Sep 10 16:23:01.488631 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:01.488613    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672\": container with ID starting with ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672 not found: ID does not exist" containerID="ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"
Sep 10 16:23:01.488708 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.488643    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672"} err="failed to get container status \"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672\": rpc error: code = NotFound desc = could not find container \"ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672\": container with ID starting with ef094cdb60c0613d30c97afaaa0662942c1411b2fd9b6d40c64966ec298b7672 not found: ID does not exist"
Sep 10 16:23:01.517826 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.517798    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:23:01.527915 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.527886    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e22c72-c6dd95dfc-6jr2x"]
Sep 10 16:23:01.545100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.545077    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qc52\" (UniqueName: \"kubernetes.io/projected/db85115f-cc59-4931-b5f3-43b84588c43e-kube-api-access-5qc52\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:01.545100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.545100    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:01.545244 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.545110    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db85115f-cc59-4931-b5f3-43b84588c43e-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:01.545244 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:01.545120    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db85115f-cc59-4931-b5f3-43b84588c43e-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:03.265900 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:03.265863    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db85115f-cc59-4931-b5f3-43b84588c43e" path="/var/lib/kubelet/pods/db85115f-cc59-4931-b5f3-43b84588c43e/volumes"
Sep 10 16:23:12.921639 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921603    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921927    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae45a17-c19f-4319-b917-48385e06c6a3" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921939    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae45a17-c19f-4319-b917-48385e06c6a3" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921947    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c400b94-f991-4931-a641-602de99c4f2e" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921953    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c400b94-f991-4931-a641-602de99c4f2e" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921962    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969aea64-77a1-441e-9d35-ea846c16cb3b" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921968    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="969aea64-77a1-441e-9d35-ea846c16cb3b" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921981    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db85115f-cc59-4931-b5f3-43b84588c43e" containerName="maas-api"
Sep 10 16:23:12.921988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921986    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db85115f-cc59-4931-b5f3-43b84588c43e" containerName="maas-api"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.921994    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f74003-6513-42e3-826b-8b28d1dbbe2e" containerName="main"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922000    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f74003-6513-42e3-826b-8b28d1dbbe2e" containerName="main"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922055    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db85115f-cc59-4931-b5f3-43b84588c43e" containerName="maas-api"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922063    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f74003-6513-42e3-826b-8b28d1dbbe2e" containerName="main"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922071    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae45a17-c19f-4319-b917-48385e06c6a3" containerName="maas-api"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922076    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c400b94-f991-4931-a641-602de99c4f2e" containerName="maas-api"
Sep 10 16:23:12.922240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.922081    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="969aea64-77a1-441e-9d35-ea846c16cb3b" containerName="maas-api"
Sep 10 16:23:12.926491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.926470    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.932175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.932148    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-664dc8\"/\"default-dockercfg-4nshp\""
Sep 10 16:23:12.932311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.932184    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-664dc8\"/\"openshift-service-ca.crt\""
Sep 10 16:23:12.932311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.932198    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-664dc8\"/\"kube-root-ca.crt\""
Sep 10 16:23:12.932311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.932158    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-664dc8\"/\"noten-model-664dc8-kserve-self-signed-certs\""
Sep 10 16:23:12.951413 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.951579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951430    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.951579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.951579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.951746 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72klb\" (UniqueName: \"kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.951746 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.951611    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:12.953523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:12.953493    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:13.052100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052561 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052525    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052658 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.052658 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72klb\" (UniqueName: \"kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.053181 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.053181 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.052929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.054565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.054538    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.054930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.054909    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.062039 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.062012    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72klb\" (UniqueName: \"kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb\") pod \"noten-model-664dc8-kserve-64865cbc5f-2xps5\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") " pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.236483 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.236384    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:13.512137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.512064    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:13.515322 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:23:13.515285    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2eb6469_64be_4801_b3e4_3e1c4fb0b2d3.slice/crio-451271360c8454d744f967b77d7b6b82c45c6162bf1f20332229a9c8f8c35719 WatchSource:0}: Error finding container 451271360c8454d744f967b77d7b6b82c45c6162bf1f20332229a9c8f8c35719: Status 404 returned error can't find the container with id 451271360c8454d744f967b77d7b6b82c45c6162bf1f20332229a9c8f8c35719
Sep 10 16:23:13.535358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:13.535312    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" event={"ID":"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3","Type":"ContainerStarted","Data":"451271360c8454d744f967b77d7b6b82c45c6162bf1f20332229a9c8f8c35719"}
Sep 10 16:23:14.192728 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.192691    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:23:14.196121 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.196101    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.199760 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.199733    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-219fd6-serving-cert\""
Sep 10 16:23:14.209728 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.209702    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:23:14.260785 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.260750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.260971 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.260856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.260971 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.260949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzhnt\" (UniqueName: \"kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.261050 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.260971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.361635 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.361595    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.362094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.362063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzhnt\" (UniqueName: \"kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.362245 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.362112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.362245 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.362161    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.362245 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:14.362220    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-serving-cert: secret "maas-api-e2e-shared-a-w3-219fd6-serving-cert" not found
Sep 10 16:23:14.362454 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:14.362282    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls podName:686b2132-cc10-4bfd-a716-d05818bfa12f nodeName:}" failed. No retries permitted until 2026-09-10 16:23:14.862262522 +0000 UTC m=+1290.095210612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls") pod "maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" (UID: "686b2132-cc10-4bfd-a716-d05818bfa12f") : secret "maas-api-e2e-shared-a-w3-219fd6-serving-cert" not found
Sep 10 16:23:14.362838 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.362814    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.364208 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.364190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.377531 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.377505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzhnt\" (UniqueName: \"kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.541607 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.541568    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" event={"ID":"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3","Type":"ContainerStarted","Data":"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"}
Sep 10 16:23:14.568981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.568919    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" podStartSLOduration=2.56889953 podStartE2EDuration="2.56889953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:23:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:14.567860652 +0000 UTC m=+1289.800808754" watchObservedRunningTime="2026-09-10 16:23:14.56889953 +0000 UTC m=+1289.801847633"
Sep 10 16:23:14.865494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.865404    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:14.867874 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:14.867849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:15.106486 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:15.106446    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:15.388515 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:15.388489    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:23:15.390414 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:23:15.390383    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686b2132_cc10_4bfd_a716_d05818bfa12f.slice/crio-fbeca256a56501d634d145d3845983e57d8274cd033ca916fc983da8b82f96c7 WatchSource:0}: Error finding container fbeca256a56501d634d145d3845983e57d8274cd033ca916fc983da8b82f96c7: Status 404 returned error can't find the container with id fbeca256a56501d634d145d3845983e57d8274cd033ca916fc983da8b82f96c7
Sep 10 16:23:15.391972 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:15.391950    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:23:15.547240 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:15.547205    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" event={"ID":"686b2132-cc10-4bfd-a716-d05818bfa12f","Type":"ContainerStarted","Data":"fbeca256a56501d634d145d3845983e57d8274cd033ca916fc983da8b82f96c7"}
Sep 10 16:23:16.553390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:16.553352    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" event={"ID":"686b2132-cc10-4bfd-a716-d05818bfa12f","Type":"ContainerStarted","Data":"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"}
Sep 10 16:23:16.553864 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:16.553450    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:16.575160 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:16.575102    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" podStartSLOduration=2.045417504 podStartE2EDuration="2.57508332s" totalImagesPullingTime="529.665816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:16.573412576 +0000 UTC m=+1291.806360678" watchObservedRunningTime="2026-09-10 16:23:16.57508332 +0000 UTC m=+1291.808031423"
Sep 10 16:23:18.551400 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:18.551363    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:22.563035 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:22.563002    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:23:23.237387 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:23.237347    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:33.571112 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.571063    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" podUID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" containerName="main" containerID="cri-o://1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b" gracePeriod=45
Sep 10 16:23:33.688661 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.688626    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:23:33.694122 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.693633    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.697569 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.697537    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-37c368a6-serving-cert\""
Sep 10 16:23:33.725725 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.725690    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:23:33.729934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.729901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.730069 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.729970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.730069 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.730014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.730069 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.730037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwfxv\" (UniqueName: \"kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.828233 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.828171    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:33.831236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.831211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.831379 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.831255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.831379 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.831283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.831379 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.831315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfxv\" (UniqueName: \"kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.831513 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:33.831387    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-serving-cert: secret "maas-api-e2e-mt-37c368a6-serving-cert" not found
Sep 10 16:23:33.831513 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:33.831453    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls podName:65153705-8adb-40ea-8cba-fb8e6dd3115b nodeName:}" failed. No retries permitted until 2026-09-10 16:23:34.331432371 +0000 UTC m=+1309.564380465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls") pod "maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" (UID: "65153705-8adb-40ea-8cba-fb8e6dd3115b") : secret "maas-api-e2e-mt-37c368a6-serving-cert" not found
Sep 10 16:23:33.831920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.831901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.833689 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.833664    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.858233 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.858206    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwfxv\" (UniqueName: \"kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:33.932296 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932267    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932307    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932351    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932371    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932401    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932432    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72klb\" (UniqueName: \"kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb\") pod \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\" (UID: \"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3\") "
Sep 10 16:23:33.932761 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932622    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home" (OuterVolumeSpecName: "home") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:33.932761 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932638    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:33.932761 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932720    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:33.932966 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932786    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:33.932966 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.932808    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:33.934683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.934656    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:23:33.934784 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.934690    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm" (OuterVolumeSpecName: "dshm") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:33.934784 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:33.934728    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb" (OuterVolumeSpecName: "kube-api-access-72klb") pod "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" (UID: "d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3"). InnerVolumeSpecName "kube-api-access-72klb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:34.033677 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.033634    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:34.033677 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.033669    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:34.033677 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.033678    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:34.033677 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.033686    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72klb\" (UniqueName: \"kubernetes.io/projected/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3-kube-api-access-72klb\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:23:34.336233 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.336190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:34.338618 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.338596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") pod \"maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:34.607104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.607011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:34.637291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.637247    2574 generic.go:356] "Generic (PLEG): container finished" podID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" containerID="1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b" exitCode=0
Sep 10 16:23:34.637291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.637292    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"
Sep 10 16:23:34.637494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.637361    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" event={"ID":"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3","Type":"ContainerDied","Data":"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"}
Sep 10 16:23:34.637494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.637410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5" event={"ID":"d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3","Type":"ContainerDied","Data":"451271360c8454d744f967b77d7b6b82c45c6162bf1f20332229a9c8f8c35719"}
Sep 10 16:23:34.637494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.637434    2574 scope.go:117] "RemoveContainer" containerID="1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"
Sep 10 16:23:34.647786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.647761    2574 scope.go:117] "RemoveContainer" containerID="1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"
Sep 10 16:23:34.648182 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:23:34.648154    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b\": container with ID starting with 1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b not found: ID does not exist" containerID="1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"
Sep 10 16:23:34.648287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.648193    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b"} err="failed to get container status \"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b\": rpc error: code = NotFound desc = could not find container \"1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b\": container with ID starting with 1a2a85078af6e8e7a23a2d5d61f106e31f919c8527090fd8c577ddf07cf1de4b not found: ID does not exist"
Sep 10 16:23:34.670874 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.670836    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:34.677355 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.677298    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-664dc8/noten-model-664dc8-kserve-64865cbc5f-2xps5"]
Sep 10 16:23:34.888892 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:34.888810    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:23:34.893546 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:23:34.893503    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65153705_8adb_40ea_8cba_fb8e6dd3115b.slice/crio-9046e5aa0f758e7d8f25d6b40690c063a5db1cedc7b230889697d1c51249c362 WatchSource:0}: Error finding container 9046e5aa0f758e7d8f25d6b40690c063a5db1cedc7b230889697d1c51249c362: Status 404 returned error can't find the container with id 9046e5aa0f758e7d8f25d6b40690c063a5db1cedc7b230889697d1c51249c362
Sep 10 16:23:35.267286 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:35.267191    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" path="/var/lib/kubelet/pods/d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3/volumes"
Sep 10 16:23:35.645244 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:35.645208    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" event={"ID":"65153705-8adb-40ea-8cba-fb8e6dd3115b","Type":"ContainerStarted","Data":"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"}
Sep 10 16:23:35.645670 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:35.645258    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" event={"ID":"65153705-8adb-40ea-8cba-fb8e6dd3115b","Type":"ContainerStarted","Data":"9046e5aa0f758e7d8f25d6b40690c063a5db1cedc7b230889697d1c51249c362"}
Sep 10 16:23:35.645670 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:35.645306    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:35.676062 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:35.676005    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" podStartSLOduration=2.289166563 podStartE2EDuration="2.675986977s" totalImagesPullingTime="386.820414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:35.67456297 +0000 UTC m=+1310.907511074" watchObservedRunningTime="2026-09-10 16:23:35.675986977 +0000 UTC m=+1310.908935079"
Sep 10 16:23:41.655754 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:41.655721    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:23:43.170986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.170941    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:23:43.171461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.171445    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" containerName="main"
Sep 10 16:23:43.171512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.171465    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" containerName="main"
Sep 10 16:23:43.171580 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.171569    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2eb6469-64be-4801-b3e4-3e1c4fb0b2d3" containerName="main"
Sep 10 16:23:43.174962 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.174937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.177932 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.177908    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-b73303-serving-cert\""
Sep 10 16:23:43.186566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.186537    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:23:43.314310 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.314275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.314524 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.314370    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgv5h\" (UniqueName: \"kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.314524 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.314427    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.314524 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.314467    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.415676 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.415639    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.415832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.415686    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.415832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.415729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgv5h\" (UniqueName: \"kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.415832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.415780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.416492 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.416467    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.418265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.418245    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.418365 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.418278    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.426139 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.426084    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgv5h\" (UniqueName: \"kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h\") pod \"maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.486808 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.486769    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:43.770544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:43.770504    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:23:43.772684 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:23:43.772651    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5475e8_d397_44de_8b1a_115982dba750.slice/crio-983e9dba1fc1361366a7fabacb51c878a5a5b0e72c01e50f38a106c81c4a6860 WatchSource:0}: Error finding container 983e9dba1fc1361366a7fabacb51c878a5a5b0e72c01e50f38a106c81c4a6860: Status 404 returned error can't find the container with id 983e9dba1fc1361366a7fabacb51c878a5a5b0e72c01e50f38a106c81c4a6860
Sep 10 16:23:44.690736 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:44.690697    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" event={"ID":"5b5475e8-d397-44de-8b1a-115982dba750","Type":"ContainerStarted","Data":"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"}
Sep 10 16:23:44.691178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:44.690791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" event={"ID":"5b5475e8-d397-44de-8b1a-115982dba750","Type":"ContainerStarted","Data":"983e9dba1fc1361366a7fabacb51c878a5a5b0e72c01e50f38a106c81c4a6860"}
Sep 10 16:23:44.691178 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:44.690942    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:23:44.727997 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:44.727929    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" podStartSLOduration=1.413154537 podStartE2EDuration="1.727909455s" totalImagesPullingTime="314.754918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:44.726071972 +0000 UTC m=+1319.959020075" watchObservedRunningTime="2026-09-10 16:23:44.727909455 +0000 UTC m=+1319.960857558"
Sep 10 16:23:50.700289 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:23:50.700255    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:24:16.314234 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.314140    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:24:16.314816 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.314426    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" podUID="65153705-8adb-40ea-8cba-fb8e6dd3115b" containerName="maas-api" containerID="cri-o://69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e" gracePeriod=30
Sep 10 16:24:16.559626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.559600    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:24:16.601993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.601910    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca\") pod \"65153705-8adb-40ea-8cba-fb8e6dd3115b\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") "
Sep 10 16:24:16.601993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.601946    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs\") pod \"65153705-8adb-40ea-8cba-fb8e6dd3115b\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") "
Sep 10 16:24:16.601993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.601991    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") pod \"65153705-8adb-40ea-8cba-fb8e6dd3115b\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") "
Sep 10 16:24:16.602258 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.602028    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwfxv\" (UniqueName: \"kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv\") pod \"65153705-8adb-40ea-8cba-fb8e6dd3115b\" (UID: \"65153705-8adb-40ea-8cba-fb8e6dd3115b\") "
Sep 10 16:24:16.602378 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.602344    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65153705-8adb-40ea-8cba-fb8e6dd3115b" (UID: "65153705-8adb-40ea-8cba-fb8e6dd3115b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:24:16.604284 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.604242    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv" (OuterVolumeSpecName: "kube-api-access-qwfxv") pod "65153705-8adb-40ea-8cba-fb8e6dd3115b" (UID: "65153705-8adb-40ea-8cba-fb8e6dd3115b"). InnerVolumeSpecName "kube-api-access-qwfxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:16.604425 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.604344    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "65153705-8adb-40ea-8cba-fb8e6dd3115b" (UID: "65153705-8adb-40ea-8cba-fb8e6dd3115b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:24:16.604525 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.604502    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65153705-8adb-40ea-8cba-fb8e6dd3115b" (UID: "65153705-8adb-40ea-8cba-fb8e6dd3115b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:24:16.703256 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.703226    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65153705-8adb-40ea-8cba-fb8e6dd3115b-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:24:16.703256 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.703254    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:24:16.703256 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.703264    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65153705-8adb-40ea-8cba-fb8e6dd3115b-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:24:16.703506 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.703271    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwfxv\" (UniqueName: \"kubernetes.io/projected/65153705-8adb-40ea-8cba-fb8e6dd3115b-kube-api-access-qwfxv\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:24:16.835666 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.835632    2574 generic.go:356] "Generic (PLEG): container finished" podID="65153705-8adb-40ea-8cba-fb8e6dd3115b" containerID="69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e" exitCode=0
Sep 10 16:24:16.835824 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.835672    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"
Sep 10 16:24:16.835824 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.835707    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" event={"ID":"65153705-8adb-40ea-8cba-fb8e6dd3115b","Type":"ContainerDied","Data":"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"}
Sep 10 16:24:16.835824 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.835762    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr" event={"ID":"65153705-8adb-40ea-8cba-fb8e6dd3115b","Type":"ContainerDied","Data":"9046e5aa0f758e7d8f25d6b40690c063a5db1cedc7b230889697d1c51249c362"}
Sep 10 16:24:16.835824 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.835785    2574 scope.go:117] "RemoveContainer" containerID="69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"
Sep 10 16:24:16.844926 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.844907    2574 scope.go:117] "RemoveContainer" containerID="69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"
Sep 10 16:24:16.845207 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:24:16.845187    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e\": container with ID starting with 69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e not found: ID does not exist" containerID="69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"
Sep 10 16:24:16.845258 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.845216    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e"} err="failed to get container status \"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e\": rpc error: code = NotFound desc = could not find container \"69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e\": container with ID starting with 69acebea9292b76b4db966500313449d0b581eb628274c6d35a94fad0daed15e not found: ID does not exist"
Sep 10 16:24:16.861310 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.861247    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:24:16.866672 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:16.866647    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-37c368a6-8c65856fd-k5qdr"]
Sep 10 16:24:17.265836 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:17.265756    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65153705-8adb-40ea-8cba-fb8e6dd3115b" path="/var/lib/kubelet/pods/65153705-8adb-40ea-8cba-fb8e6dd3115b/volumes"
Sep 10 16:24:24.623458 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.623415    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:24:24.624034 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.623925    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65153705-8adb-40ea-8cba-fb8e6dd3115b" containerName="maas-api"
Sep 10 16:24:24.624034 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.623946    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="65153705-8adb-40ea-8cba-fb8e6dd3115b" containerName="maas-api"
Sep 10 16:24:24.624154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.624044    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="65153705-8adb-40ea-8cba-fb8e6dd3115b" containerName="maas-api"
Sep 10 16:24:24.628365 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.628320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.632860 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.632836    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"kube-root-ca.crt\""
Sep 10 16:24:24.634202 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.634176    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"openshift-service-ca.crt\""
Sep 10 16:24:24.634407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.634216    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"default-dockercfg-9vcfw\""
Sep 10 16:24:24.634407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.634186    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"auth-test-model-219fd6-kserve-self-signed-certs\""
Sep 10 16:24:24.649377 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.649348    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:24:24.673673 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.673636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.673952 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.673923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.674074 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.673987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.674141 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.674111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.674199 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.674150    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.674268 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.674246    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krm28\" (UniqueName: \"kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775221 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775307    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krm28\" (UniqueName: \"kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775435 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775396    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775722 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775722 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775707    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.775802 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.775747    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.777529 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.777507    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.777892 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.777873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.785102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.785066    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krm28\" (UniqueName: \"kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28\") pod \"auth-test-model-219fd6-kserve-5b8b5db799-5rzpz\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:24.938115 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:24.938029    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:25.214353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:25.214315    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:24:25.216580 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:24:25.216540    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4486919_1c5e_40d8_8690_6546c41c18aa.slice/crio-c181fb9ea1c5f4e3a28768b4571e672c75241f30ed14e6c75e483f880a31c872 WatchSource:0}: Error finding container c181fb9ea1c5f4e3a28768b4571e672c75241f30ed14e6c75e483f880a31c872: Status 404 returned error can't find the container with id c181fb9ea1c5f4e3a28768b4571e672c75241f30ed14e6c75e483f880a31c872
Sep 10 16:24:25.878436 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:25.878373    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" event={"ID":"d4486919-1c5e-40d8-8690-6546c41c18aa","Type":"ContainerStarted","Data":"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"}
Sep 10 16:24:25.878839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:25.878452    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" event={"ID":"d4486919-1c5e-40d8-8690-6546c41c18aa","Type":"ContainerStarted","Data":"c181fb9ea1c5f4e3a28768b4571e672c75241f30ed14e6c75e483f880a31c872"}
Sep 10 16:24:25.898850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:25.898500    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" podStartSLOduration=1.898479655 podStartE2EDuration="1.898479655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:25.897443649 +0000 UTC m=+1361.130391752" watchObservedRunningTime="2026-09-10 16:24:25.898479655 +0000 UTC m=+1361.131427758"
Sep 10 16:24:34.938615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:34.938575    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:34.939106 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:34.938636    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:34.951374 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:34.951348    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:35.936112 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:35.936074    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:24:43.949168 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:43.949134    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:24:43.952610 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:43.952590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:43.959946 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:43.959741    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-29fd7d-serving-cert\""
Sep 10 16:24:43.970188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:43.970146    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:24:44.051853 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.051806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.051853 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.051854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.052104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.051872    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpw8f\" (UniqueName: \"kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.052104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.051980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.153175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.153129    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.153175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.153175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.153485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.153196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpw8f\" (UniqueName: \"kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.153485 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.153246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.153485 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:24:44.153279    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-serving-cert: secret "maas-api-e2e-shared-a-w2-29fd7d-serving-cert" not found
Sep 10 16:24:44.153485 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:24:44.153393    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls podName:bf9a6cf1-085d-4224-ac91-fe8665d18003 nodeName:}" failed. No retries permitted until 2026-09-10 16:24:44.653371662 +0000 UTC m=+1379.886319767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls") pod "maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" (UID: "bf9a6cf1-085d-4224-ac91-fe8665d18003") : secret "maas-api-e2e-shared-a-w2-29fd7d-serving-cert" not found
Sep 10 16:24:44.153854 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.153833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.155736 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.155716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.175214 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.175184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpw8f\" (UniqueName: \"kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.658684 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.658643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.661346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.661303    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:44.865816 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:44.865773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:45.150511 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:45.150481    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:24:45.151111 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:24:45.151086    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9a6cf1_085d_4224_ac91_fe8665d18003.slice/crio-fc6fcc91fe4ec761dfaac7534b49e1ce2b23200eb74674f94abe9fd6287bbdee WatchSource:0}: Error finding container fc6fcc91fe4ec761dfaac7534b49e1ce2b23200eb74674f94abe9fd6287bbdee: Status 404 returned error can't find the container with id fc6fcc91fe4ec761dfaac7534b49e1ce2b23200eb74674f94abe9fd6287bbdee
Sep 10 16:24:45.970967 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:45.970893    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" event={"ID":"bf9a6cf1-085d-4224-ac91-fe8665d18003","Type":"ContainerStarted","Data":"fc6fcc91fe4ec761dfaac7534b49e1ce2b23200eb74674f94abe9fd6287bbdee"}
Sep 10 16:24:46.977064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:46.977028    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" event={"ID":"bf9a6cf1-085d-4224-ac91-fe8665d18003","Type":"ContainerStarted","Data":"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"}
Sep 10 16:24:46.977542 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:46.977089    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:24:47.016356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:47.016277    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" podStartSLOduration=3.231340477 podStartE2EDuration="4.016261239s" totalImagesPullingTime="784.920762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:47.015588268 +0000 UTC m=+1382.248536370" watchObservedRunningTime="2026-09-10 16:24:47.016261239 +0000 UTC m=+1382.249209341"
Sep 10 16:24:52.987988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:24:52.987956    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:25:12.606846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.606800    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:25:12.610692 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.610672    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.613927 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.613889    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-90bd7e-serving-cert\""
Sep 10 16:25:12.622049 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.622018    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:25:12.696241 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.696202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjqvh\" (UniqueName: \"kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.696443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.696288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.696443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.696313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.696443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.696345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.797193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.797158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.797193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.797198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.797415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.797218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.797415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.797266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjqvh\" (UniqueName: \"kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.797415 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:12.797371    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-serving-cert: secret "maas-api-e2e-shared-b-w2-90bd7e-serving-cert" not found
Sep 10 16:25:12.797514 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:12.797469    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls podName:1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0 nodeName:}" failed. No retries permitted until 2026-09-10 16:25:13.2974462 +0000 UTC m=+1408.530394280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls") pod "maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" (UID: "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0") : secret "maas-api-e2e-shared-b-w2-90bd7e-serving-cert" not found
Sep 10 16:25:12.797794 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.797775    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.799730 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.799713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:12.811415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:12.811387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjqvh\" (UniqueName: \"kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:13.301215 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:13.301167    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:13.303615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:13.303592    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:13.522593 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:13.522552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:13.803894 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:13.803866    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:25:13.805941 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:13.805909    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4e3bdf_1a3b_4b1a_8950_b5d0f1ffbbc0.slice/crio-32c3727ed1b64475769a99d7833994652c5d187d68071834569839331f425196 WatchSource:0}: Error finding container 32c3727ed1b64475769a99d7833994652c5d187d68071834569839331f425196: Status 404 returned error can't find the container with id 32c3727ed1b64475769a99d7833994652c5d187d68071834569839331f425196
Sep 10 16:25:14.110888 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:14.110847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" event={"ID":"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0","Type":"ContainerStarted","Data":"32c3727ed1b64475769a99d7833994652c5d187d68071834569839331f425196"}
Sep 10 16:25:15.116628 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:15.116594    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" event={"ID":"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0","Type":"ContainerStarted","Data":"af8f9f910bf0b82e1b50e70eee7988175259fa3486e322b0a75b275a349e1cf1"}
Sep 10 16:25:15.117006 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:15.116729    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:15.138620 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:15.138564    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" podStartSLOduration=2.830465074 podStartE2EDuration="3.138547383s" totalImagesPullingTime="308.082309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:15.137147215 +0000 UTC m=+1410.370095318" watchObservedRunningTime="2026-09-10 16:25:15.138547383 +0000 UTC m=+1410.371495508"
Sep 10 16:25:21.126819 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:21.126786    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:25:37.082719 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.082680    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-90ab93"]
Sep 10 16:25:37.086097 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.086078    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:37.089052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.089020    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8f29h\""
Sep 10 16:25:37.092011 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.091979    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-90ab93"]
Sep 10 16:25:37.233741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.233707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6hgz\" (UniqueName: \"kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz\") pod \"test-curl-7462-90ab93\" (UID: \"43ad0a93-9d6e-45bd-baa6-22af6c193907\") " pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:37.335250 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.335149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hgz\" (UniqueName: \"kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz\") pod \"test-curl-7462-90ab93\" (UID: \"43ad0a93-9d6e-45bd-baa6-22af6c193907\") " pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:37.347806 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.347780    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hgz\" (UniqueName: \"kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz\") pod \"test-curl-7462-90ab93\" (UID: \"43ad0a93-9d6e-45bd-baa6-22af6c193907\") " pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:37.396880 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.396836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:37.668261 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:37.668236    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-90ab93"]
Sep 10 16:25:37.670955 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:37.670922    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43ad0a93_9d6e_45bd_baa6_22af6c193907.slice/crio-c4f46187d5212da0987ebf5acc84a8d1fa1b996d15391bff844d7d43ca1f83b2 WatchSource:0}: Error finding container c4f46187d5212da0987ebf5acc84a8d1fa1b996d15391bff844d7d43ca1f83b2: Status 404 returned error can't find the container with id c4f46187d5212da0987ebf5acc84a8d1fa1b996d15391bff844d7d43ca1f83b2
Sep 10 16:25:38.231127 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:38.231089    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-90ab93" event={"ID":"43ad0a93-9d6e-45bd-baa6-22af6c193907","Type":"ContainerStarted","Data":"c4f46187d5212da0987ebf5acc84a8d1fa1b996d15391bff844d7d43ca1f83b2"}
Sep 10 16:25:40.245903 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:40.245866    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-90ab93" event={"ID":"43ad0a93-9d6e-45bd-baa6-22af6c193907","Type":"ContainerStarted","Data":"407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575"}
Sep 10 16:25:40.914947 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:40.914885    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-90ab93" podStartSLOduration=1.985108039 podStartE2EDuration="3.914868342s" totalImagesPullingTime="1.929760303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:40.266235227 +0000 UTC m=+1435.499183328" watchObservedRunningTime="2026-09-10 16:25:40.914868342 +0000 UTC m=+1436.147816444"
Sep 10 16:25:40.916531 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:40.916503    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-90ab93"]
Sep 10 16:25:40.923151 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:40.923118    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-90ab93"]
Sep 10 16:25:41.058588 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.058550    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-836e66"]
Sep 10 16:25:41.058947 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.058934    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43ad0a93-9d6e-45bd-baa6-22af6c193907" containerName="test-curl-7462-90ab93"
Sep 10 16:25:41.058992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.058949    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43ad0a93-9d6e-45bd-baa6-22af6c193907" containerName="test-curl-7462-90ab93"
Sep 10 16:25:41.059032 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.059004    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43ad0a93-9d6e-45bd-baa6-22af6c193907" containerName="test-curl-7462-90ab93"
Sep 10 16:25:41.061997 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.061977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:41.069765 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.069735    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-836e66"]
Sep 10 16:25:41.172509 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.172414    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvvrp\" (UniqueName: \"kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp\") pod \"test-curl-7462-836e66\" (UID: \"218b44b6-6c19-433c-9de9-f0638443ebb2\") " pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:41.250490 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:41.250457    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-90ab93" container="test-curl-7462-90ab93"
Sep 10 16:25:41.273146 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.273116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvvrp\" (UniqueName: \"kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp\") pod \"test-curl-7462-836e66\" (UID: \"218b44b6-6c19-433c-9de9-f0638443ebb2\") " pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:41.287782 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.287755    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvvrp\" (UniqueName: \"kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp\") pod \"test-curl-7462-836e66\" (UID: \"218b44b6-6c19-433c-9de9-f0638443ebb2\") " pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:41.372374 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.372300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:41.644512 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:41.644486    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-836e66"]
Sep 10 16:25:41.646671 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:41.646647    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod218b44b6_6c19_433c_9de9_f0638443ebb2.slice/crio-308fa2b81d4099382e13ae6c03f3d193e7cb74c83809a491c075c284dc15d2c2 WatchSource:0}: Error finding container 308fa2b81d4099382e13ae6c03f3d193e7cb74c83809a491c075c284dc15d2c2: Status 404 returned error can't find the container with id 308fa2b81d4099382e13ae6c03f3d193e7cb74c83809a491c075c284dc15d2c2
Sep 10 16:25:42.256615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:42.256577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-836e66" event={"ID":"218b44b6-6c19-433c-9de9-f0638443ebb2","Type":"ContainerStarted","Data":"4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37"}
Sep 10 16:25:42.256615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:42.256619    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-836e66" event={"ID":"218b44b6-6c19-433c-9de9-f0638443ebb2","Type":"ContainerStarted","Data":"308fa2b81d4099382e13ae6c03f3d193e7cb74c83809a491c075c284dc15d2c2"}
Sep 10 16:25:42.257105 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:42.256752    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-90ab93" podUID="43ad0a93-9d6e-45bd-baa6-22af6c193907" containerName="test-curl-7462-90ab93" containerID="cri-o://407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575" gracePeriod=2
Sep 10 16:25:42.285837 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:42.285782    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-836e66" podStartSLOduration=1.285766644 podStartE2EDuration="1.285766644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:42.285308785 +0000 UTC m=+1437.518256887" watchObservedRunningTime="2026-09-10 16:25:42.285766644 +0000 UTC m=+1437.518714746"
Sep 10 16:25:43.147016 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.146931    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-836e66"]
Sep 10 16:25:43.153692 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.153661    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-836e66"]
Sep 10 16:25:43.261858 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:43.261822    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-836e66" container="test-curl-7462-836e66"
Sep 10 16:25:43.291013 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.290981    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-fbb883"]
Sep 10 16:25:43.291342 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.291315    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218b44b6-6c19-433c-9de9-f0638443ebb2" containerName="test-curl-7462-836e66"
Sep 10 16:25:43.291389 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.291345    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="218b44b6-6c19-433c-9de9-f0638443ebb2" containerName="test-curl-7462-836e66"
Sep 10 16:25:43.291424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.291403    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="218b44b6-6c19-433c-9de9-f0638443ebb2" containerName="test-curl-7462-836e66"
Sep 10 16:25:43.294492 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.294474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:43.304292 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.304265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-fbb883"]
Sep 10 16:25:43.394045 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.394001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwv84\" (UniqueName: \"kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84\") pod \"test-curl-7462-fbb883\" (UID: \"9db234f0-2ec9-44e2-8c7c-12a7a00dea92\") " pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:43.495357 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.495258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwv84\" (UniqueName: \"kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84\") pod \"test-curl-7462-fbb883\" (UID: \"9db234f0-2ec9-44e2-8c7c-12a7a00dea92\") " pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:43.509172 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.509147    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwv84\" (UniqueName: \"kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84\") pod \"test-curl-7462-fbb883\" (UID: \"9db234f0-2ec9-44e2-8c7c-12a7a00dea92\") " pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:43.604052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.604017    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:43.885299 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:43.885269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-fbb883"]
Sep 10 16:25:43.886868 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:43.886836    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9db234f0_2ec9_44e2_8c7c_12a7a00dea92.slice/crio-a98a331f1327c02b0c64417cfa3f9df360f84d69f42813ae651865b4fb6f18e0 WatchSource:0}: Error finding container a98a331f1327c02b0c64417cfa3f9df360f84d69f42813ae651865b4fb6f18e0: Status 404 returned error can't find the container with id a98a331f1327c02b0c64417cfa3f9df360f84d69f42813ae651865b4fb6f18e0
Sep 10 16:25:44.267879 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.267852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-fbb883" event={"ID":"9db234f0-2ec9-44e2-8c7c-12a7a00dea92","Type":"ContainerStarted","Data":"bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5"}
Sep 10 16:25:44.268173 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.267889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-fbb883" event={"ID":"9db234f0-2ec9-44e2-8c7c-12a7a00dea92","Type":"ContainerStarted","Data":"a98a331f1327c02b0c64417cfa3f9df360f84d69f42813ae651865b4fb6f18e0"}
Sep 10 16:25:44.268173 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.267899    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-836e66" podUID="218b44b6-6c19-433c-9de9-f0638443ebb2" containerName="test-curl-7462-836e66" containerID="cri-o://4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37" gracePeriod=2
Sep 10 16:25:44.291751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.291689    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-fbb883" podStartSLOduration=1.291675289 podStartE2EDuration="1.291675289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:44.289110099 +0000 UTC m=+1439.522058195" watchObservedRunningTime="2026-09-10 16:25:44.291675289 +0000 UTC m=+1439.524623391"
Sep 10 16:25:44.424875 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.424842    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:44.503846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.503814    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6hgz\" (UniqueName: \"kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz\") pod \"43ad0a93-9d6e-45bd-baa6-22af6c193907\" (UID: \"43ad0a93-9d6e-45bd-baa6-22af6c193907\") "
Sep 10 16:25:44.505847 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.505819    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz" (OuterVolumeSpecName: "kube-api-access-q6hgz") pod "43ad0a93-9d6e-45bd-baa6-22af6c193907" (UID: "43ad0a93-9d6e-45bd-baa6-22af6c193907"). InnerVolumeSpecName "kube-api-access-q6hgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:44.604621 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.604588    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6hgz\" (UniqueName: \"kubernetes.io/projected/43ad0a93-9d6e-45bd-baa6-22af6c193907-kube-api-access-q6hgz\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:25:44.926211 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.926174    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-fbb883"]
Sep 10 16:25:44.934945 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:44.934910    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-fbb883"]
Sep 10 16:25:45.061356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.061309    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-f1fec9"]
Sep 10 16:25:45.061713 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.061699    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" containerName="test-curl-7462-fbb883"
Sep 10 16:25:45.061765 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.061714    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" containerName="test-curl-7462-fbb883"
Sep 10 16:25:45.061800 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.061776    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" containerName="test-curl-7462-fbb883"
Sep 10 16:25:45.064743 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.064725    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:45.070862 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.070830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-f1fec9"]
Sep 10 16:25:45.107301 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.107257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hw6\" (UniqueName: \"kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6\") pod \"test-curl-7462-f1fec9\" (UID: \"8c46b2bd-d12f-4bd7-8797-17032621c2ba\") " pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:45.208356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.208265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hw6\" (UniqueName: \"kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6\") pod \"test-curl-7462-f1fec9\" (UID: \"8c46b2bd-d12f-4bd7-8797-17032621c2ba\") " pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:45.220711 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.220675    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hw6\" (UniqueName: \"kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6\") pod \"test-curl-7462-f1fec9\" (UID: \"8c46b2bd-d12f-4bd7-8797-17032621c2ba\") " pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:45.266786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.266754    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43ad0a93-9d6e-45bd-baa6-22af6c193907" path="/var/lib/kubelet/pods/43ad0a93-9d6e-45bd-baa6-22af6c193907/volumes"
Sep 10 16:25:45.274018 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.273990    2574 generic.go:356] "Generic (PLEG): container finished" podID="43ad0a93-9d6e-45bd-baa6-22af6c193907" containerID="407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575" exitCode=137
Sep 10 16:25:45.274386 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.274032    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-90ab93"
Sep 10 16:25:45.274386 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.274086    2574 scope.go:117] "RemoveContainer" containerID="407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575"
Sep 10 16:25:45.274386 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:45.274272    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-fbb883" container="test-curl-7462-fbb883"
Sep 10 16:25:45.283195 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.283179    2574 scope.go:117] "RemoveContainer" containerID="407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575"
Sep 10 16:25:45.283547 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:45.283529    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575\": container with ID starting with 407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575 not found: ID does not exist" containerID="407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575"
Sep 10 16:25:45.283605 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.283553    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575"} err="failed to get container status \"407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575\": rpc error: code = NotFound desc = could not find container \"407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575\": container with ID starting with 407b5375a97bdaac2cf69d846f38e32f53b46c49be35cf50ff1ae83a03402575 not found: ID does not exist"
Sep 10 16:25:45.376634 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.376596    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:45.649766 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:45.649733    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-f1fec9"]
Sep 10 16:25:45.652902 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:45.652872    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c46b2bd_d12f_4bd7_8797_17032621c2ba.slice/crio-9e5aea6600d3a54bc22625341d728e368d1cdac75c129724abfa511aa9218bd1 WatchSource:0}: Error finding container 9e5aea6600d3a54bc22625341d728e368d1cdac75c129724abfa511aa9218bd1: Status 404 returned error can't find the container with id 9e5aea6600d3a54bc22625341d728e368d1cdac75c129724abfa511aa9218bd1
Sep 10 16:25:46.281826 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.281797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-f1fec9" event={"ID":"8c46b2bd-d12f-4bd7-8797-17032621c2ba","Type":"ContainerStarted","Data":"2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8"}
Sep 10 16:25:46.282236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.281842    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-f1fec9" event={"ID":"8c46b2bd-d12f-4bd7-8797-17032621c2ba","Type":"ContainerStarted","Data":"9e5aea6600d3a54bc22625341d728e368d1cdac75c129724abfa511aa9218bd1"}
Sep 10 16:25:46.282236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.281942    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-fbb883" podUID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" containerName="test-curl-7462-fbb883" containerID="cri-o://bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5" gracePeriod=2
Sep 10 16:25:46.302262 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.302204    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-f1fec9" podStartSLOduration=1.302189084 podStartE2EDuration="1.302189084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:46.300578659 +0000 UTC m=+1441.533526775" watchObservedRunningTime="2026-09-10 16:25:46.302189084 +0000 UTC m=+1441.535137185"
Sep 10 16:25:46.405085 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.405057    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:46.418341 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.418309    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvvrp\" (UniqueName: \"kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp\") pod \"218b44b6-6c19-433c-9de9-f0638443ebb2\" (UID: \"218b44b6-6c19-433c-9de9-f0638443ebb2\") "
Sep 10 16:25:46.420259 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.420231    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp" (OuterVolumeSpecName: "kube-api-access-bvvrp") pod "218b44b6-6c19-433c-9de9-f0638443ebb2" (UID: "218b44b6-6c19-433c-9de9-f0638443ebb2"). InnerVolumeSpecName "kube-api-access-bvvrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:46.519494 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.519462    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvvrp\" (UniqueName: \"kubernetes.io/projected/218b44b6-6c19-433c-9de9-f0638443ebb2-kube-api-access-bvvrp\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:25:46.958835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.958744    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-f1fec9"]
Sep 10 16:25:46.970114 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:46.970081    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-f1fec9"]
Sep 10 16:25:47.266500 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.266412    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="218b44b6-6c19-433c-9de9-f0638443ebb2" path="/var/lib/kubelet/pods/218b44b6-6c19-433c-9de9-f0638443ebb2/volumes"
Sep 10 16:25:47.287346 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.287297    2574 generic.go:356] "Generic (PLEG): container finished" podID="218b44b6-6c19-433c-9de9-f0638443ebb2" containerID="4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37" exitCode=137
Sep 10 16:25:47.287741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.287353    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-836e66"
Sep 10 16:25:47.287741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.287379    2574 scope.go:117] "RemoveContainer" containerID="4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37"
Sep 10 16:25:47.287741 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:47.287600    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7462-f1fec9" container="test-curl-7462-f1fec9"
Sep 10 16:25:47.295876 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.295857    2574 scope.go:117] "RemoveContainer" containerID="4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37"
Sep 10 16:25:47.296145 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:47.296125    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37\": container with ID starting with 4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37 not found: ID does not exist" containerID="4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37"
Sep 10 16:25:47.296194 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:47.296154    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37"} err="failed to get container status \"4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37\": rpc error: code = NotFound desc = could not find container \"4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37\": container with ID starting with 4711d11a6199ab9d4af22edbb2db7141975334ceaff17506f9e4c6e97f146f37 not found: ID does not exist"
Sep 10 16:25:48.293792 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:48.293757    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7462-f1fec9" podUID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" containerName="test-curl-7462-f1fec9" containerID="cri-o://2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8" gracePeriod=2
Sep 10 16:25:48.469004 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:48.468972    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:48.536185 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:48.536106    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwv84\" (UniqueName: \"kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84\") pod \"9db234f0-2ec9-44e2-8c7c-12a7a00dea92\" (UID: \"9db234f0-2ec9-44e2-8c7c-12a7a00dea92\") "
Sep 10 16:25:48.538070 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:48.538044    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84" (OuterVolumeSpecName: "kube-api-access-qwv84") pod "9db234f0-2ec9-44e2-8c7c-12a7a00dea92" (UID: "9db234f0-2ec9-44e2-8c7c-12a7a00dea92"). InnerVolumeSpecName "kube-api-access-qwv84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:48.637267 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:48.637228    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwv84\" (UniqueName: \"kubernetes.io/projected/9db234f0-2ec9-44e2-8c7c-12a7a00dea92-kube-api-access-qwv84\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:25:49.266616 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.266582    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" path="/var/lib/kubelet/pods/9db234f0-2ec9-44e2-8c7c-12a7a00dea92/volumes"
Sep 10 16:25:49.299665 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.299630    2574 generic.go:356] "Generic (PLEG): container finished" podID="9db234f0-2ec9-44e2-8c7c-12a7a00dea92" containerID="bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5" exitCode=137
Sep 10 16:25:49.300100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.299672    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-fbb883"
Sep 10 16:25:49.300100 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.299758    2574 scope.go:117] "RemoveContainer" containerID="bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5"
Sep 10 16:25:49.308171 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.308151    2574 scope.go:117] "RemoveContainer" containerID="bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5"
Sep 10 16:25:49.308503 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:49.308477    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5\": container with ID starting with bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5 not found: ID does not exist" containerID="bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5"
Sep 10 16:25:49.308572 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:49.308514    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5"} err="failed to get container status \"bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5\": rpc error: code = NotFound desc = could not find container \"bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5\": container with ID starting with bfdf9e792980339cd563d0d078e77390758a77ae1dcc0d74536bd0afed6eeaa5 not found: ID does not exist"
Sep 10 16:25:50.482596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:50.482568    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:50.552918 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:50.552819    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4hw6\" (UniqueName: \"kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6\") pod \"8c46b2bd-d12f-4bd7-8797-17032621c2ba\" (UID: \"8c46b2bd-d12f-4bd7-8797-17032621c2ba\") "
Sep 10 16:25:50.554918 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:50.554892    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6" (OuterVolumeSpecName: "kube-api-access-q4hw6") pod "8c46b2bd-d12f-4bd7-8797-17032621c2ba" (UID: "8c46b2bd-d12f-4bd7-8797-17032621c2ba"). InnerVolumeSpecName "kube-api-access-q4hw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:50.654490 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:50.654451    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hw6\" (UniqueName: \"kubernetes.io/projected/8c46b2bd-d12f-4bd7-8797-17032621c2ba-kube-api-access-q4hw6\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:25:51.266127 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.266093    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" path="/var/lib/kubelet/pods/8c46b2bd-d12f-4bd7-8797-17032621c2ba/volumes"
Sep 10 16:25:51.312219 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.312188    2574 generic.go:356] "Generic (PLEG): container finished" podID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" containerID="2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8" exitCode=137
Sep 10 16:25:51.312445 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.312232    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-f1fec9"
Sep 10 16:25:51.312445 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.312239    2574 scope.go:117] "RemoveContainer" containerID="2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8"
Sep 10 16:25:51.320507 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.320480    2574 scope.go:117] "RemoveContainer" containerID="2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8"
Sep 10 16:25:51.320781 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:51.320760    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8\": container with ID starting with 2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8 not found: ID does not exist" containerID="2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8"
Sep 10 16:25:51.320837 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.320789    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8"} err="failed to get container status \"2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8\": rpc error: code = NotFound desc = could not find container \"2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8\": container with ID starting with 2e06c47b9aa1d121ae3412686ddbc21e19664d491527b7dfbc2d1bf0ec9d29b8 not found: ID does not exist"
Sep 10 16:25:51.917413 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.917381    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:25:51.917851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.917718    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" containerName="test-curl-7462-f1fec9"
Sep 10 16:25:51.917851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.917730    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" containerName="test-curl-7462-f1fec9"
Sep 10 16:25:51.917851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.917790    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c46b2bd-d12f-4bd7-8797-17032621c2ba" containerName="test-curl-7462-f1fec9"
Sep 10 16:25:51.922631 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.922611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:51.925978 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.925955    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7dabb3eb-serving-cert\""
Sep 10 16:25:51.936827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.936802    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:25:51.968242 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.968210    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhfx7\" (UniqueName: \"kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:51.968430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.968266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:51.968430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.968290    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:51.968430 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:51.968352    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.069846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.069806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhfx7\" (UniqueName: \"kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.070023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.069868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.070023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.069890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.070023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.069907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.070133 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:52.070033    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-serving-cert: secret "maas-api-e2e-mt-7dabb3eb-serving-cert" not found
Sep 10 16:25:52.070133 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:25:52.070114    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls podName:ff9cb96c-6d57-47ab-92c0-78061d93d361 nodeName:}" failed. No retries permitted until 2026-09-10 16:25:52.570092529 +0000 UTC m=+1447.803040611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls") pod "maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" (UID: "ff9cb96c-6d57-47ab-92c0-78061d93d361") : secret "maas-api-e2e-mt-7dabb3eb-serving-cert" not found
Sep 10 16:25:52.070641 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.070617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.072522 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.072493    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.081971 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.081950    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhfx7\" (UniqueName: \"kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.574483 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.574444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.577059 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.577028    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") pod \"maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:52.833468 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:52.833378    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:53.113635 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:53.113605    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:25:53.116892 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:25:53.116861    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9cb96c_6d57_47ab_92c0_78061d93d361.slice/crio-e98a6979540dccbbed8434ebbf688bfc38ce833d6791e90f15208b031abf7a25 WatchSource:0}: Error finding container e98a6979540dccbbed8434ebbf688bfc38ce833d6791e90f15208b031abf7a25: Status 404 returned error can't find the container with id e98a6979540dccbbed8434ebbf688bfc38ce833d6791e90f15208b031abf7a25
Sep 10 16:25:53.324102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:53.324028    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" event={"ID":"ff9cb96c-6d57-47ab-92c0-78061d93d361","Type":"ContainerStarted","Data":"e98a6979540dccbbed8434ebbf688bfc38ce833d6791e90f15208b031abf7a25"}
Sep 10 16:25:54.330644 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:54.330603    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" event={"ID":"ff9cb96c-6d57-47ab-92c0-78061d93d361","Type":"ContainerStarted","Data":"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"}
Sep 10 16:25:54.331118 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:54.330722    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:25:54.353130 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:25:54.353075    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" podStartSLOduration=2.987517151 podStartE2EDuration="3.353061813s" totalImagesPullingTime="365.544662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:54.351393326 +0000 UTC m=+1449.584341427" watchObservedRunningTime="2026-09-10 16:25:54.353061813 +0000 UTC m=+1449.586009915"
Sep 10 16:26:00.340175 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:26:00.340142    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:26:45.283926 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:26:45.283891    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:26:45.285541 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:26:45.285518    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:27:08.516444 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.516352    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-93b3d0"]
Sep 10 16:27:08.519857 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.519836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:08.522748 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.522724    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8f29h\""
Sep 10 16:27:08.527778 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.527754    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-93b3d0"]
Sep 10 16:27:08.672152 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.672117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76mk6\" (UniqueName: \"kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6\") pod \"test-curl-7465-93b3d0\" (UID: \"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e\") " pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:08.773150 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.773059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76mk6\" (UniqueName: \"kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6\") pod \"test-curl-7465-93b3d0\" (UID: \"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e\") " pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:08.783828 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.783793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76mk6\" (UniqueName: \"kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6\") pod \"test-curl-7465-93b3d0\" (UID: \"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e\") " pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:08.829669 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:08.829628    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:09.102775 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:09.102749    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-93b3d0"]
Sep 10 16:27:09.105240 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:27:09.105208    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae96c4a7_6945_4b2e_b9be_f57ef9902a7e.slice/crio-8d28dfaa5d8d960076d399b649c1335ae4fe31122532333088cd5ac672ba47d7 WatchSource:0}: Error finding container 8d28dfaa5d8d960076d399b649c1335ae4fe31122532333088cd5ac672ba47d7: Status 404 returned error can't find the container with id 8d28dfaa5d8d960076d399b649c1335ae4fe31122532333088cd5ac672ba47d7
Sep 10 16:27:09.701983 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:09.701947    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-93b3d0" event={"ID":"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e","Type":"ContainerStarted","Data":"bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b"}
Sep 10 16:27:09.701983 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:09.701990    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-93b3d0" event={"ID":"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e","Type":"ContainerStarted","Data":"8d28dfaa5d8d960076d399b649c1335ae4fe31122532333088cd5ac672ba47d7"}
Sep 10 16:27:09.724145 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:09.724092    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7465-93b3d0" podStartSLOduration=1.724075644 podStartE2EDuration="1.724075644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:27:09.722421269 +0000 UTC m=+1524.955369372" watchObservedRunningTime="2026-09-10 16:27:09.724075644 +0000 UTC m=+1524.957023746"
Sep 10 16:27:10.370120 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:10.370079    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-93b3d0"]
Sep 10 16:27:10.377886 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:10.377853    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-93b3d0"]
Sep 10 16:27:10.707018 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:10.706986    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7465-93b3d0" container="test-curl-7465-93b3d0"
Sep 10 16:27:10.914477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:10.914444    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:27:10.914691 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:10.914662    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" podUID="ff9cb96c-6d57-47ab-92c0-78061d93d361" containerName="maas-api" containerID="cri-o://951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4" gracePeriod=30
Sep 10 16:27:11.169250 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.169224    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:27:11.189981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.189944    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") pod \"ff9cb96c-6d57-47ab-92c0-78061d93d361\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") "
Sep 10 16:27:11.190252 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.189989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca\") pod \"ff9cb96c-6d57-47ab-92c0-78061d93d361\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") "
Sep 10 16:27:11.190252 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.190043    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhfx7\" (UniqueName: \"kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7\") pod \"ff9cb96c-6d57-47ab-92c0-78061d93d361\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") "
Sep 10 16:27:11.190252 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.190123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs\") pod \"ff9cb96c-6d57-47ab-92c0-78061d93d361\" (UID: \"ff9cb96c-6d57-47ab-92c0-78061d93d361\") "
Sep 10 16:27:11.190508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.190468    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff9cb96c-6d57-47ab-92c0-78061d93d361" (UID: "ff9cb96c-6d57-47ab-92c0-78061d93d361"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:27:11.192218 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.192181    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff9cb96c-6d57-47ab-92c0-78061d93d361" (UID: "ff9cb96c-6d57-47ab-92c0-78061d93d361"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:27:11.192370 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.192226    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7" (OuterVolumeSpecName: "kube-api-access-qhfx7") pod "ff9cb96c-6d57-47ab-92c0-78061d93d361" (UID: "ff9cb96c-6d57-47ab-92c0-78061d93d361"). InnerVolumeSpecName "kube-api-access-qhfx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:27:11.192370 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.192227    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ff9cb96c-6d57-47ab-92c0-78061d93d361" (UID: "ff9cb96c-6d57-47ab-92c0-78061d93d361"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:27:11.291158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.291075    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhfx7\" (UniqueName: \"kubernetes.io/projected/ff9cb96c-6d57-47ab-92c0-78061d93d361-kube-api-access-qhfx7\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:11.291158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.291108    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:11.291158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.291118    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9cb96c-6d57-47ab-92c0-78061d93d361-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:11.291158 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.291126    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cb96c-6d57-47ab-92c0-78061d93d361-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:11.712970 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.712881    2574 generic.go:356] "Generic (PLEG): container finished" podID="ff9cb96c-6d57-47ab-92c0-78061d93d361" containerID="951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4" exitCode=0
Sep 10 16:27:11.712970 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.712927    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"
Sep 10 16:27:11.713471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.712964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" event={"ID":"ff9cb96c-6d57-47ab-92c0-78061d93d361","Type":"ContainerDied","Data":"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"}
Sep 10 16:27:11.713471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.713019    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr" event={"ID":"ff9cb96c-6d57-47ab-92c0-78061d93d361","Type":"ContainerDied","Data":"e98a6979540dccbbed8434ebbf688bfc38ce833d6791e90f15208b031abf7a25"}
Sep 10 16:27:11.713471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.713050    2574 scope.go:117] "RemoveContainer" containerID="951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"
Sep 10 16:27:11.713471 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.713142    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7465-93b3d0" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" containerName="test-curl-7465-93b3d0" containerID="cri-o://bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b" gracePeriod=2
Sep 10 16:27:11.721845 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.721826    2574 scope.go:117] "RemoveContainer" containerID="951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"
Sep 10 16:27:11.722102 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:11.722082    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4\": container with ID starting with 951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4 not found: ID does not exist" containerID="951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"
Sep 10 16:27:11.722172 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.722116    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4"} err="failed to get container status \"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4\": rpc error: code = NotFound desc = could not find container \"951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4\": container with ID starting with 951b1d0a6557459b709fd8bfca811f2c7a5e81669fd3ad1054492ca7d1242ef4 not found: ID does not exist"
Sep 10 16:27:11.754502 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.754468    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:27:11.769124 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:11.769090    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7dabb3eb-78465dbd49-wk6wr"]
Sep 10 16:27:12.748344 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:12.748287    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:27:13.268830 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:13.268796    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9cb96c-6d57-47ab-92c0-78061d93d361" path="/var/lib/kubelet/pods/ff9cb96c-6d57-47ab-92c0-78061d93d361/volumes"
Sep 10 16:27:13.855853 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:13.855828    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:13.915285 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:13.915255    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76mk6\" (UniqueName: \"kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6\") pod \"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e\" (UID: \"ae96c4a7-6945-4b2e-b9be-f57ef9902a7e\") "
Sep 10 16:27:13.917254 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:13.917222    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6" (OuterVolumeSpecName: "kube-api-access-76mk6") pod "ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" (UID: "ae96c4a7-6945-4b2e-b9be-f57ef9902a7e"). InnerVolumeSpecName "kube-api-access-76mk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:27:14.016390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.016270    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76mk6\" (UniqueName: \"kubernetes.io/projected/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e-kube-api-access-76mk6\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:14.732772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.732734    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" containerID="bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b" exitCode=137
Sep 10 16:27:14.732966 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.732775    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:14.732966 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.732793    2574 scope.go:117] "RemoveContainer" containerID="bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b"
Sep 10 16:27:14.741142 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.741123    2574 scope.go:117] "RemoveContainer" containerID="bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b"
Sep 10 16:27:14.741477 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:14.741455    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b\": container with ID starting with bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b not found: ID does not exist" containerID="bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b"
Sep 10 16:27:14.741566 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:14.741486    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b"} err="failed to get container status \"bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b\": rpc error: code = NotFound desc = could not find container \"bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b\": container with ID starting with bc0a8450ebb18e39307b6975a49259dfe2ed85b1dcd042904fd8e319ceaf7e9b not found: ID does not exist"
Sep 10 16:27:14.743395 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:14.743373    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-93b3d0\" is forbidden: User \"system:node:ip-10-0-143-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-169.ec2.internal' and this object" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:15.265639 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:15.265601    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-93b3d0\" is forbidden: User \"system:node:ip-10-0-143-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-169.ec2.internal' and this object" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" pod="opendatahub/test-curl-7465-93b3d0"
Sep 10 16:27:15.267204 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:15.267173    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" path="/var/lib/kubelet/pods/ae96c4a7-6945-4b2e-b9be-f57ef9902a7e/volumes"
Sep 10 16:27:24.807972 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.807932    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:27:24.808381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808279    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9cb96c-6d57-47ab-92c0-78061d93d361" containerName="maas-api"
Sep 10 16:27:24.808381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808292    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9cb96c-6d57-47ab-92c0-78061d93d361" containerName="maas-api"
Sep 10 16:27:24.808381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808301    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" containerName="test-curl-7465-93b3d0"
Sep 10 16:27:24.808381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808307    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" containerName="test-curl-7465-93b3d0"
Sep 10 16:27:24.808523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808391    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae96c4a7-6945-4b2e-b9be-f57ef9902a7e" containerName="test-curl-7465-93b3d0"
Sep 10 16:27:24.808523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.808402    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9cb96c-6d57-47ab-92c0-78061d93d361" containerName="maas-api"
Sep 10 16:27:24.811257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.811240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.814023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.813999    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"sub-test-model-219fd6-kserve-self-signed-certs\""
Sep 10 16:27:24.823976 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.823946    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:27:24.917969 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.917931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.917969 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.917970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.918189 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.918085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.918189 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.918118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.918189 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.918142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:24.918292 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:24.918217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6dt8\" (UniqueName: \"kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019662 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019662 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.019920 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.019778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6dt8\" (UniqueName: \"kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.020128 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.020092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.020196 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.020126    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.020196 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.020139    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.022124 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.022098    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.022287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.022268    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.029627 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.029584    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6dt8\" (UniqueName: \"kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8\") pod \"sub-test-model-219fd6-kserve-859f444c87-rlst6\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.120725 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.120638    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:25.393981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.393924    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:27:25.396926 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:27:25.396899    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf650416b_6918_4883_9467_f15824dd4570.slice/crio-9bb98bf30712155f11a813c84e2df52a179c65378633be3fad7ef62cab7740bb WatchSource:0}: Error finding container 9bb98bf30712155f11a813c84e2df52a179c65378633be3fad7ef62cab7740bb: Status 404 returned error can't find the container with id 9bb98bf30712155f11a813c84e2df52a179c65378633be3fad7ef62cab7740bb
Sep 10 16:27:25.785537 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.785500    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" event={"ID":"f650416b-6918-4883-9467-f15824dd4570","Type":"ContainerStarted","Data":"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"}
Sep 10 16:27:25.785708 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.785553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" event={"ID":"f650416b-6918-4883-9467-f15824dd4570","Type":"ContainerStarted","Data":"9bb98bf30712155f11a813c84e2df52a179c65378633be3fad7ef62cab7740bb"}
Sep 10 16:27:25.809224 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:25.809170    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" podStartSLOduration=1.8091550939999999 podStartE2EDuration="1.809155094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:27:25.806712327 +0000 UTC m=+1541.039660430" watchObservedRunningTime="2026-09-10 16:27:25.809155094 +0000 UTC m=+1541.042103196"
Sep 10 16:27:27.767866 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:27.767809    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" podUID="d4486919-1c5e-40d8-8690-6546c41c18aa" containerName="main" containerID="cri-o://2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75" gracePeriod=45
Sep 10 16:27:28.020051 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.019986    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:27:28.150371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150308    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150417    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150449    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150586 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150549    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krm28\" (UniqueName: \"kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28\") pod \"d4486919-1c5e-40d8-8690-6546c41c18aa\" (UID: \"d4486919-1c5e-40d8-8690-6546c41c18aa\") "
Sep 10 16:27:28.150861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150753    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:27:28.150861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150764    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home" (OuterVolumeSpecName: "home") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:27:28.150861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150830    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:27:28.150998 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150919    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.150998 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150937    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.150998 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.150949    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.152800 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.152772    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm" (OuterVolumeSpecName: "dshm") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:27:28.152911 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.152803    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28" (OuterVolumeSpecName: "kube-api-access-krm28") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "kube-api-access-krm28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:27:28.152911 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.152811    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4486919-1c5e-40d8-8690-6546c41c18aa" (UID: "d4486919-1c5e-40d8-8690-6546c41c18aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:27:28.252065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.252020    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krm28\" (UniqueName: \"kubernetes.io/projected/d4486919-1c5e-40d8-8690-6546c41c18aa-kube-api-access-krm28\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.252065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.252060    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4486919-1c5e-40d8-8690-6546c41c18aa-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.252065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.252071    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4486919-1c5e-40d8-8690-6546c41c18aa-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:27:28.803030 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.802988    2574 generic.go:356] "Generic (PLEG): container finished" podID="d4486919-1c5e-40d8-8690-6546c41c18aa" containerID="2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75" exitCode=0
Sep 10 16:27:28.803525 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.803043    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"
Sep 10 16:27:28.803525 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.803069    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" event={"ID":"d4486919-1c5e-40d8-8690-6546c41c18aa","Type":"ContainerDied","Data":"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"}
Sep 10 16:27:28.803525 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.803106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz" event={"ID":"d4486919-1c5e-40d8-8690-6546c41c18aa","Type":"ContainerDied","Data":"c181fb9ea1c5f4e3a28768b4571e672c75241f30ed14e6c75e483f880a31c872"}
Sep 10 16:27:28.803525 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.803122    2574 scope.go:117] "RemoveContainer" containerID="2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"
Sep 10 16:27:28.813542 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.813519    2574 scope.go:117] "RemoveContainer" containerID="2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"
Sep 10 16:27:28.813843 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:27:28.813822    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75\": container with ID starting with 2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75 not found: ID does not exist" containerID="2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"
Sep 10 16:27:28.813891 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.813853    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75"} err="failed to get container status \"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75\": rpc error: code = NotFound desc = could not find container \"2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75\": container with ID starting with 2133a5dce3134eb4b2e710670d8ffcba15c89d7115bead22cf9e30110115fa75 not found: ID does not exist"
Sep 10 16:27:28.828104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.828072    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:27:28.835665 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:28.835633    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/auth-test-model-219fd6-kserve-5b8b5db799-5rzpz"]
Sep 10 16:27:29.267934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:29.267856    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4486919-1c5e-40d8-8690-6546c41c18aa" path="/var/lib/kubelet/pods/d4486919-1c5e-40d8-8690-6546c41c18aa/volumes"
Sep 10 16:27:35.121587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:35.121552    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:35.121982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:35.121604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:35.134106 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:35.134071    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:27:35.847251 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:27:35.847220    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:28:32.499663 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.499627    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-410a97"]
Sep 10 16:28:32.500154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.499958    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4486919-1c5e-40d8-8690-6546c41c18aa" containerName="main"
Sep 10 16:28:32.500154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.499971    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4486919-1c5e-40d8-8690-6546c41c18aa" containerName="main"
Sep 10 16:28:32.500154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.500039    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4486919-1c5e-40d8-8690-6546c41c18aa" containerName="main"
Sep 10 16:28:32.503123 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.503103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:32.505784 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.505754    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8f29h\""
Sep 10 16:28:32.510104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.510053    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-410a97"]
Sep 10 16:28:32.631859 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.631819    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbvff\" (UniqueName: \"kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff\") pod \"test-curl-7465-410a97\" (UID: \"eec46a81-d7cd-49a0-b477-9cf0755b8a1c\") " pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:32.733225 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.733194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbvff\" (UniqueName: \"kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff\") pod \"test-curl-7465-410a97\" (UID: \"eec46a81-d7cd-49a0-b477-9cf0755b8a1c\") " pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:32.750622 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.750565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbvff\" (UniqueName: \"kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff\") pod \"test-curl-7465-410a97\" (UID: \"eec46a81-d7cd-49a0-b477-9cf0755b8a1c\") " pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:32.813416 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:32.813369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:33.084917 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:33.084891    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-410a97"]
Sep 10 16:28:33.086779 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:28:33.086748    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeec46a81_d7cd_49a0_b477_9cf0755b8a1c.slice/crio-a4dfcc94241fa1049d68183293cb187fe02a42f9cc14b865f62b7c4f7240b23c WatchSource:0}: Error finding container a4dfcc94241fa1049d68183293cb187fe02a42f9cc14b865f62b7c4f7240b23c: Status 404 returned error can't find the container with id a4dfcc94241fa1049d68183293cb187fe02a42f9cc14b865f62b7c4f7240b23c
Sep 10 16:28:33.109530 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:33.109504    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-410a97" event={"ID":"eec46a81-d7cd-49a0-b477-9cf0755b8a1c","Type":"ContainerStarted","Data":"a4dfcc94241fa1049d68183293cb187fe02a42f9cc14b865f62b7c4f7240b23c"}
Sep 10 16:28:34.115829 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.115792    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-410a97" event={"ID":"eec46a81-d7cd-49a0-b477-9cf0755b8a1c","Type":"ContainerStarted","Data":"7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df"}
Sep 10 16:28:34.134493 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.134437    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7465-410a97" podStartSLOduration=2.134421181 podStartE2EDuration="2.134421181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:28:34.132954252 +0000 UTC m=+1609.365902355" watchObservedRunningTime="2026-09-10 16:28:34.134421181 +0000 UTC m=+1609.367369283"
Sep 10 16:28:34.733448 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.733414    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-410a97"]
Sep 10 16:28:34.739231 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.739197    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-410a97"]
Sep 10 16:28:34.855521 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.855488    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-793b96"]
Sep 10 16:28:34.855835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.855822    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" containerName="test-curl-7465-410a97"
Sep 10 16:28:34.855878 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.855837    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" containerName="test-curl-7465-410a97"
Sep 10 16:28:34.855917 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.855905    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" containerName="test-curl-7465-410a97"
Sep 10 16:28:34.858617 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.858601    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:34.866402 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.866376    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-793b96"]
Sep 10 16:28:34.953539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:34.953503    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vhtl\" (UniqueName: \"kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl\") pod \"test-curl-7465-793b96\" (UID: \"2499af43-012e-49ab-84f0-0e054aa1fffa\") " pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:35.054452 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:35.054414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vhtl\" (UniqueName: \"kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl\") pod \"test-curl-7465-793b96\" (UID: \"2499af43-012e-49ab-84f0-0e054aa1fffa\") " pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:35.065836 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:35.065798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vhtl\" (UniqueName: \"kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl\") pod \"test-curl-7465-793b96\" (UID: \"2499af43-012e-49ab-84f0-0e054aa1fffa\") " pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:35.120515 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:35.120444    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7465-410a97" container="test-curl-7465-410a97"
Sep 10 16:28:35.168238 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:35.168207    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:35.441905 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:35.441827    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-793b96"]
Sep 10 16:28:35.445151 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:28:35.445121    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2499af43_012e_49ab_84f0_0e054aa1fffa.slice/crio-2291caf3a1323a04d8be59207dec9d1f46eff3790029a06b3d8c6beb8cd96ffb WatchSource:0}: Error finding container 2291caf3a1323a04d8be59207dec9d1f46eff3790029a06b3d8c6beb8cd96ffb: Status 404 returned error can't find the container with id 2291caf3a1323a04d8be59207dec9d1f46eff3790029a06b3d8c6beb8cd96ffb
Sep 10 16:28:36.126718 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.126678    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-793b96" event={"ID":"2499af43-012e-49ab-84f0-0e054aa1fffa","Type":"ContainerStarted","Data":"67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce"}
Sep 10 16:28:36.127155 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.126730    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-793b96" event={"ID":"2499af43-012e-49ab-84f0-0e054aa1fffa","Type":"ContainerStarted","Data":"2291caf3a1323a04d8be59207dec9d1f46eff3790029a06b3d8c6beb8cd96ffb"}
Sep 10 16:28:36.127155 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.126755    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7465-410a97" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" containerName="test-curl-7465-410a97" containerID="cri-o://7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df" gracePeriod=2
Sep 10 16:28:36.145744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.145696    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7465-793b96" podStartSLOduration=2.145682033 podStartE2EDuration="2.145682033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:28:36.143707201 +0000 UTC m=+1611.376655308" watchObservedRunningTime="2026-09-10 16:28:36.145682033 +0000 UTC m=+1611.378630171"
Sep 10 16:28:36.763085 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.763047    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-793b96"]
Sep 10 16:28:36.772858 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:36.772824    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-793b96"]
Sep 10 16:28:37.132485 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:37.132450    2574 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7465-793b96" container="test-curl-7465-793b96"
Sep 10 16:28:38.137104 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.137070    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7465-793b96" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" containerName="test-curl-7465-793b96" containerID="cri-o://67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce" gracePeriod=2
Sep 10 16:28:38.298059 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.298028    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:38.376481 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.376451    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbvff\" (UniqueName: \"kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff\") pod \"eec46a81-d7cd-49a0-b477-9cf0755b8a1c\" (UID: \"eec46a81-d7cd-49a0-b477-9cf0755b8a1c\") "
Sep 10 16:28:38.378558 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.378530    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff" (OuterVolumeSpecName: "kube-api-access-wbvff") pod "eec46a81-d7cd-49a0-b477-9cf0755b8a1c" (UID: "eec46a81-d7cd-49a0-b477-9cf0755b8a1c"). InnerVolumeSpecName "kube-api-access-wbvff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:28:38.477046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.477006    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbvff\" (UniqueName: \"kubernetes.io/projected/eec46a81-d7cd-49a0-b477-9cf0755b8a1c-kube-api-access-wbvff\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:38.842832 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:38.842798    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:28:39.143670 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.143580    2574 generic.go:356] "Generic (PLEG): container finished" podID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" containerID="7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df" exitCode=137
Sep 10 16:28:39.143670 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.143618    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:39.143670 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.143655    2574 scope.go:117] "RemoveContainer" containerID="7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df"
Sep 10 16:28:39.152521 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.152501    2574 scope.go:117] "RemoveContainer" containerID="7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df"
Sep 10 16:28:39.152831 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:39.152810    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df\": container with ID starting with 7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df not found: ID does not exist" containerID="7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df"
Sep 10 16:28:39.152910 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.152842    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df"} err="failed to get container status \"7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df\": rpc error: code = NotFound desc = could not find container \"7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df\": container with ID starting with 7952de13b8dd1c42b60dad50d375f60289bb9efffdbdf41d1e9ae6d68f4be6df not found: ID does not exist"
Sep 10 16:28:39.154369 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:39.154346    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-410a97\" is forbidden: User \"system:node:ip-10-0-143-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-169.ec2.internal' and this object" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" pod="opendatahub/test-curl-7465-410a97"
Sep 10 16:28:39.266102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:39.266069    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eec46a81-d7cd-49a0-b477-9cf0755b8a1c" path="/var/lib/kubelet/pods/eec46a81-d7cd-49a0-b477-9cf0755b8a1c/volumes"
Sep 10 16:28:40.284553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:40.284529    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:40.289659 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:40.289643    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vhtl\" (UniqueName: \"kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl\") pod \"2499af43-012e-49ab-84f0-0e054aa1fffa\" (UID: \"2499af43-012e-49ab-84f0-0e054aa1fffa\") "
Sep 10 16:28:40.291598 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:40.291571    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl" (OuterVolumeSpecName: "kube-api-access-5vhtl") pod "2499af43-012e-49ab-84f0-0e054aa1fffa" (UID: "2499af43-012e-49ab-84f0-0e054aa1fffa"). InnerVolumeSpecName "kube-api-access-5vhtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:28:40.391063 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:40.390987    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vhtl\" (UniqueName: \"kubernetes.io/projected/2499af43-012e-49ab-84f0-0e054aa1fffa-kube-api-access-5vhtl\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:41.155681 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.155646    2574 generic.go:356] "Generic (PLEG): container finished" podID="2499af43-012e-49ab-84f0-0e054aa1fffa" containerID="67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce" exitCode=137
Sep 10 16:28:41.155864 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.155685    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:41.155864 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.155750    2574 scope.go:117] "RemoveContainer" containerID="67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce"
Sep 10 16:28:41.164744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.164567    2574 scope.go:117] "RemoveContainer" containerID="67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce"
Sep 10 16:28:41.164821 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:41.164801    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce\": container with ID starting with 67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce not found: ID does not exist" containerID="67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce"
Sep 10 16:28:41.164875 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.164830    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce"} err="failed to get container status \"67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce\": rpc error: code = NotFound desc = could not find container \"67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce\": container with ID starting with 67824008dfb15409c722ef251677dc505040b27789ac4d84b46b298b9043f0ce not found: ID does not exist"
Sep 10 16:28:41.167798 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:41.167779    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-793b96\" is forbidden: User \"system:node:ip-10-0-143-169.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-169.ec2.internal' and this object" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" pod="opendatahub/test-curl-7465-793b96"
Sep 10 16:28:41.266276 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:41.266246    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" path="/var/lib/kubelet/pods/2499af43-012e-49ab-84f0-0e054aa1fffa/volumes"
Sep 10 16:28:49.725449 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.725413    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:28:49.725827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.725749    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" containerName="test-curl-7465-793b96"
Sep 10 16:28:49.725827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.725761    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" containerName="test-curl-7465-793b96"
Sep 10 16:28:49.725827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.725828    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2499af43-012e-49ab-84f0-0e054aa1fffa" containerName="test-curl-7465-793b96"
Sep 10 16:28:49.728648 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.728629    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.733806 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.733783    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-219fd6\"/\"rate-test-model-219fd6-kserve-self-signed-certs\""
Sep 10 16:28:49.747115 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.747089    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:28:49.774639 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774616    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.774771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.774771 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.774888 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.774888 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.775009 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.774884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spl5d\" (UniqueName: \"kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.875872 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875840    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spl5d\" (UniqueName: \"kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875955    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876038 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.875969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.876382    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.876387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.876477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.876433    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.878078 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.878057    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.878391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.878374    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:49.888528 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:49.888503    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spl5d\" (UniqueName: \"kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d\") pod \"rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") " pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:50.037140 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:50.037111    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:28:50.311156 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:50.311132    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:28:50.313399 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:28:50.313369    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6ef3672_986f_4b65_b40d_8c4ae133d66c.slice/crio-e67578cfa9a41d910cca9289a2edd52d9f9a7b59521d1c208cc6523722165029 WatchSource:0}: Error finding container e67578cfa9a41d910cca9289a2edd52d9f9a7b59521d1c208cc6523722165029: Status 404 returned error can't find the container with id e67578cfa9a41d910cca9289a2edd52d9f9a7b59521d1c208cc6523722165029
Sep 10 16:28:51.206250 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:51.206212    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" event={"ID":"a6ef3672-986f-4b65-b40d-8c4ae133d66c","Type":"ContainerStarted","Data":"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"}
Sep 10 16:28:51.206751 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:51.206261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" event={"ID":"a6ef3672-986f-4b65-b40d-8c4ae133d66c","Type":"ContainerStarted","Data":"e67578cfa9a41d910cca9289a2edd52d9f9a7b59521d1c208cc6523722165029"}
Sep 10 16:28:51.230689 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:51.230641    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" podStartSLOduration=2.230627502 podStartE2EDuration="2.230627502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:28:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:28:51.22826038 +0000 UTC m=+1626.461208483" watchObservedRunningTime="2026-09-10 16:28:51.230627502 +0000 UTC m=+1626.463575603"
Sep 10 16:28:53.861828 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:53.861769    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" podUID="f650416b-6918-4883-9467-f15824dd4570" containerName="main" containerID="cri-o://63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e" gracePeriod=45
Sep 10 16:28:54.114603 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.114552    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:28:54.218358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6dt8\" (UniqueName: \"kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218440    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218539 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218535    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218747 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218562    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir\") pod \"f650416b-6918-4883-9467-f15824dd4570\" (UID: \"f650416b-6918-4883-9467-f15824dd4570\") "
Sep 10 16:28:54.218747 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218705    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache" (OuterVolumeSpecName: "model-cache") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:28:54.218852 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218741    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home" (OuterVolumeSpecName: "home") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:28:54.218852 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218824    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:54.218852 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218842    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:54.219007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.218920    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:28:54.220687 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.220656    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8" (OuterVolumeSpecName: "kube-api-access-h6dt8") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "kube-api-access-h6dt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:28:54.220687 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.220670    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm" (OuterVolumeSpecName: "dshm") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:28:54.220687 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.220679    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f650416b-6918-4883-9467-f15824dd4570" (UID: "f650416b-6918-4883-9467-f15824dd4570"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:28:54.222091 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.222065    2574 generic.go:356] "Generic (PLEG): container finished" podID="f650416b-6918-4883-9467-f15824dd4570" containerID="63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e" exitCode=0
Sep 10 16:28:54.222193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.222105    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"
Sep 10 16:28:54.222193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.222136    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" event={"ID":"f650416b-6918-4883-9467-f15824dd4570","Type":"ContainerDied","Data":"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"}
Sep 10 16:28:54.222193 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.222178    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6" event={"ID":"f650416b-6918-4883-9467-f15824dd4570","Type":"ContainerDied","Data":"9bb98bf30712155f11a813c84e2df52a179c65378633be3fad7ef62cab7740bb"}
Sep 10 16:28:54.222309 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.222195    2574 scope.go:117] "RemoveContainer" containerID="63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"
Sep 10 16:28:54.239168 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.239151    2574 scope.go:117] "RemoveContainer" containerID="63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"
Sep 10 16:28:54.239453 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:28:54.239432    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e\": container with ID starting with 63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e not found: ID does not exist" containerID="63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"
Sep 10 16:28:54.239506 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.239460    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e"} err="failed to get container status \"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e\": rpc error: code = NotFound desc = could not find container \"63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e\": container with ID starting with 63f561045c00152eb15fcff67e9d9b4fd3ab82bfe68d46d70bc7eec0c444827e not found: ID does not exist"
Sep 10 16:28:54.253852 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.253828    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:28:54.260474 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.260452    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/sub-test-model-219fd6-kserve-859f444c87-rlst6"]
Sep 10 16:28:54.319960 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.319936    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6dt8\" (UniqueName: \"kubernetes.io/projected/f650416b-6918-4883-9467-f15824dd4570-kube-api-access-h6dt8\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:54.320096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.319963    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:54.320096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.319977    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f650416b-6918-4883-9467-f15824dd4570-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:54.320096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:54.319990    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f650416b-6918-4883-9467-f15824dd4570-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:28:55.265908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:28:55.265873    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f650416b-6918-4883-9467-f15824dd4570" path="/var/lib/kubelet/pods/f650416b-6918-4883-9467-f15824dd4570/volumes"
Sep 10 16:29:00.038095 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:00.038055    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:29:00.038095 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:00.038104    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:29:00.050692 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:00.050664    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:29:00.260457 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:00.260422    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:29:06.205437 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.205403    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:29:06.205889 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.205742    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f650416b-6918-4883-9467-f15824dd4570" containerName="main"
Sep 10 16:29:06.205889 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.205754    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f650416b-6918-4883-9467-f15824dd4570" containerName="main"
Sep 10 16:29:06.205889 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.205809    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f650416b-6918-4883-9467-f15824dd4570" containerName="main"
Sep 10 16:29:06.210053 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.210032    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.213113 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.213085    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ab677e-serving-cert\""
Sep 10 16:29:06.219794 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.219768    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:29:06.322054 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.322021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhlc\" (UniqueName: \"kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.322228 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.322073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.322228 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.322159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.322369 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.322310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.423004 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.422976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.423154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.423043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhlc\" (UniqueName: \"kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.423154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.423079    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.423154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.423106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.423807 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.423783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.425585 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.425557    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.425671 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.425598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.433643 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.433615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhlc\" (UniqueName: \"kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc\") pod \"maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.521257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.521175    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:06.796754 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.796725    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:29:06.798754 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:29:06.798725    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426cb99c_81bc_46b8_8a9e_62178e87078f.slice/crio-2b4e289bb1682eaa13ee9d42efc12582544ed1ca57aa0a883e53589872b26e93 WatchSource:0}: Error finding container 2b4e289bb1682eaa13ee9d42efc12582544ed1ca57aa0a883e53589872b26e93: Status 404 returned error can't find the container with id 2b4e289bb1682eaa13ee9d42efc12582544ed1ca57aa0a883e53589872b26e93
Sep 10 16:29:06.800305 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:06.800285    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:29:07.284242 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:07.284213    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" event={"ID":"426cb99c-81bc-46b8-8a9e-62178e87078f","Type":"ContainerStarted","Data":"2b4e289bb1682eaa13ee9d42efc12582544ed1ca57aa0a883e53589872b26e93"}
Sep 10 16:29:08.290621 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:08.290583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" event={"ID":"426cb99c-81bc-46b8-8a9e-62178e87078f","Type":"ContainerStarted","Data":"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"}
Sep 10 16:29:08.291032 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:08.290746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:08.319515 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:08.319465    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" podStartSLOduration=1.865594062 podStartE2EDuration="2.319448582s" totalImagesPullingTime="453.85452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:08.317959999 +0000 UTC m=+1643.550908104" watchObservedRunningTime="2026-09-10 16:29:08.319448582 +0000 UTC m=+1643.552396684"
Sep 10 16:29:14.301372 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:14.301321    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:29:35.277101 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.277063    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:29:35.280760 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.280736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.284475 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.284449    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cde4a2-serving-cert\""
Sep 10 16:29:35.293683 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.293654    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:29:35.397414 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.397377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.397601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.397421    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.397601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.397443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26q6\" (UniqueName: \"kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.397601 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.397490    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.498407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.498370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.498407 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.498413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.498679 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.498432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26q6\" (UniqueName: \"kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.498679 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.498452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.499170 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.499148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.500959 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.500939    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.501064 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.500975    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.509075 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.509044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26q6\" (UniqueName: \"kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6\") pod \"maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.593657 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.593568    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:35.869387 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:35.869359    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:29:35.871463 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:29:35.871435    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea48c037_05c7_4b9e_ac9a_0e06480482a9.slice/crio-abbc73a376f862a1db1ecac66924985ed47d6356e9914ced4a1030413e267439 WatchSource:0}: Error finding container abbc73a376f862a1db1ecac66924985ed47d6356e9914ced4a1030413e267439: Status 404 returned error can't find the container with id abbc73a376f862a1db1ecac66924985ed47d6356e9914ced4a1030413e267439
Sep 10 16:29:36.424629 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:36.424591    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" event={"ID":"ea48c037-05c7-4b9e-ac9a-0e06480482a9","Type":"ContainerStarted","Data":"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"}
Sep 10 16:29:36.425090 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:36.424648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" event={"ID":"ea48c037-05c7-4b9e-ac9a-0e06480482a9","Type":"ContainerStarted","Data":"abbc73a376f862a1db1ecac66924985ed47d6356e9914ced4a1030413e267439"}
Sep 10 16:29:36.425090 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:36.424802    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:29:36.450535 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:36.450471    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" podStartSLOduration=1.051745549 podStartE2EDuration="1.45045229s" totalImagesPullingTime="398.706741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:36.447919005 +0000 UTC m=+1671.680867106" watchObservedRunningTime="2026-09-10 16:29:36.45045229 +0000 UTC m=+1671.683400392"
Sep 10 16:29:42.434094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:29:42.434064    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:30:18.031748 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.031669    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:30:18.032224 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.031910    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" podUID="426cb99c-81bc-46b8-8a9e-62178e87078f" containerName="maas-api" containerID="cri-o://b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd" gracePeriod=30
Sep 10 16:30:18.279513 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.279482    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:30:18.405272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.405194    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzhlc\" (UniqueName: \"kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc\") pod \"426cb99c-81bc-46b8-8a9e-62178e87078f\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") "
Sep 10 16:30:18.405272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.405225    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls\") pod \"426cb99c-81bc-46b8-8a9e-62178e87078f\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") "
Sep 10 16:30:18.405272 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.405267    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs\") pod \"426cb99c-81bc-46b8-8a9e-62178e87078f\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") "
Sep 10 16:30:18.405593 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.405292    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca\") pod \"426cb99c-81bc-46b8-8a9e-62178e87078f\" (UID: \"426cb99c-81bc-46b8-8a9e-62178e87078f\") "
Sep 10 16:30:18.405732 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.405700    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "426cb99c-81bc-46b8-8a9e-62178e87078f" (UID: "426cb99c-81bc-46b8-8a9e-62178e87078f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:30:18.407486 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.407462    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "426cb99c-81bc-46b8-8a9e-62178e87078f" (UID: "426cb99c-81bc-46b8-8a9e-62178e87078f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:18.407486 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.407475    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc" (OuterVolumeSpecName: "kube-api-access-dzhlc") pod "426cb99c-81bc-46b8-8a9e-62178e87078f" (UID: "426cb99c-81bc-46b8-8a9e-62178e87078f"). InnerVolumeSpecName "kube-api-access-dzhlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:30:18.407607 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.407486    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "426cb99c-81bc-46b8-8a9e-62178e87078f" (UID: "426cb99c-81bc-46b8-8a9e-62178e87078f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:18.506304 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.506270    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzhlc\" (UniqueName: \"kubernetes.io/projected/426cb99c-81bc-46b8-8a9e-62178e87078f-kube-api-access-dzhlc\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:18.506304 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.506298    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:18.506304 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.506307    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/426cb99c-81bc-46b8-8a9e-62178e87078f-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:18.506571 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.506315    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/426cb99c-81bc-46b8-8a9e-62178e87078f-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:18.621243 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.621208    2574 generic.go:356] "Generic (PLEG): container finished" podID="426cb99c-81bc-46b8-8a9e-62178e87078f" containerID="b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd" exitCode=0
Sep 10 16:30:18.621429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.621254    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"
Sep 10 16:30:18.621429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.621295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" event={"ID":"426cb99c-81bc-46b8-8a9e-62178e87078f","Type":"ContainerDied","Data":"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"}
Sep 10 16:30:18.621429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.621365    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d" event={"ID":"426cb99c-81bc-46b8-8a9e-62178e87078f","Type":"ContainerDied","Data":"2b4e289bb1682eaa13ee9d42efc12582544ed1ca57aa0a883e53589872b26e93"}
Sep 10 16:30:18.621429 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.621395    2574 scope.go:117] "RemoveContainer" containerID="b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"
Sep 10 16:30:18.630183 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.630164    2574 scope.go:117] "RemoveContainer" containerID="b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"
Sep 10 16:30:18.630465 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:30:18.630442    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd\": container with ID starting with b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd not found: ID does not exist" containerID="b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"
Sep 10 16:30:18.630538 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.630472    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd"} err="failed to get container status \"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd\": rpc error: code = NotFound desc = could not find container \"b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd\": container with ID starting with b70183a0b2814045ba4421e7e7039bb32354aa151566c93760f04d2764dc08bd not found: ID does not exist"
Sep 10 16:30:18.660207 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.660150    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:30:18.663975 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:18.663949    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ab677e-6f685bd684-l9g8d"]
Sep 10 16:30:19.266531 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:19.266497    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426cb99c-81bc-46b8-8a9e-62178e87078f" path="/var/lib/kubelet/pods/426cb99c-81bc-46b8-8a9e-62178e87078f/volumes"
Sep 10 16:30:22.354652 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:22.354614    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:30:27.871655 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.871619    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:30:27.872135 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.871955    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426cb99c-81bc-46b8-8a9e-62178e87078f" containerName="maas-api"
Sep 10 16:30:27.872135 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.871966    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="426cb99c-81bc-46b8-8a9e-62178e87078f" containerName="maas-api"
Sep 10 16:30:27.872135 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.872038    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="426cb99c-81bc-46b8-8a9e-62178e87078f" containerName="maas-api"
Sep 10 16:30:27.876340 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.876305    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:27.880041 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.880015    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-814d40-serving-cert\""
Sep 10 16:30:27.888624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.888599    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:30:27.989993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.989956    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:27.989993 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.989993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:27.990212 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.990045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkcr\" (UniqueName: \"kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:27.990212 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:27.990075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.091188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.091154    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.091188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.091189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.091459 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.091222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkcr\" (UniqueName: \"kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.091459 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.091243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.091956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.091927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.093731 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.093713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.093810 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.093794    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.102938 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.102913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkcr\" (UniqueName: \"kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr\") pod \"maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.188356 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.188246    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:28.475788 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.475758    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:30:28.478037 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:30:28.477997    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ed8e54_d109_4da7_81f4_a871af667316.slice/crio-7faa7987463874fe994c27faf23a5f78f93241f090afef4f5d758373b6f6ee70 WatchSource:0}: Error finding container 7faa7987463874fe994c27faf23a5f78f93241f090afef4f5d758373b6f6ee70: Status 404 returned error can't find the container with id 7faa7987463874fe994c27faf23a5f78f93241f090afef4f5d758373b6f6ee70
Sep 10 16:30:28.669495 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:28.669456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" event={"ID":"a8ed8e54-d109-4da7-81f4-a871af667316","Type":"ContainerStarted","Data":"7faa7987463874fe994c27faf23a5f78f93241f090afef4f5d758373b6f6ee70"}
Sep 10 16:30:29.676070 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:29.676031    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" event={"ID":"a8ed8e54-d109-4da7-81f4-a871af667316","Type":"ContainerStarted","Data":"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"}
Sep 10 16:30:29.676556 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:29.676163    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:29.700477 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:29.700427    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" podStartSLOduration=2.107677769 podStartE2EDuration="2.700411477s" totalImagesPullingTime="592.733708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:30:29.698784298 +0000 UTC m=+1724.931732400" watchObservedRunningTime="2026-09-10 16:30:29.700411477 +0000 UTC m=+1724.933359579"
Sep 10 16:30:35.686257 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:35.686225    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:30:37.374111 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.374062    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" podUID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" containerName="main" containerID="cri-o://da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913" gracePeriod=45
Sep 10 16:30:37.627198 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.627132    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:30:37.713150 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.713112    2574 generic.go:356] "Generic (PLEG): container finished" podID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" containerID="da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913" exitCode=0
Sep 10 16:30:37.713353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.713156    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"
Sep 10 16:30:37.713353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.713197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" event={"ID":"a6ef3672-986f-4b65-b40d-8c4ae133d66c","Type":"ContainerDied","Data":"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"}
Sep 10 16:30:37.713353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.713243    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h" event={"ID":"a6ef3672-986f-4b65-b40d-8c4ae133d66c","Type":"ContainerDied","Data":"e67578cfa9a41d910cca9289a2edd52d9f9a7b59521d1c208cc6523722165029"}
Sep 10 16:30:37.713353 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.713264    2574 scope.go:117] "RemoveContainer" containerID="da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"
Sep 10 16:30:37.722004 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.721986    2574 scope.go:117] "RemoveContainer" containerID="da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"
Sep 10 16:30:37.722280 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:30:37.722259    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913\": container with ID starting with da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913 not found: ID does not exist" containerID="da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"
Sep 10 16:30:37.722358 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.722290    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913"} err="failed to get container status \"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913\": rpc error: code = NotFound desc = could not find container \"da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913\": container with ID starting with da15db912aa8b26e34ebc42c0dbc765d9fc8a834fc8a329f08c63edb88cda913 not found: ID does not exist"
Sep 10 16:30:37.782734 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782703    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.782930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782775    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spl5d\" (UniqueName: \"kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.782930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782816    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.782930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782842    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.782930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782891    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.783136 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782972    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache\") pod \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\" (UID: \"a6ef3672-986f-4b65-b40d-8c4ae133d66c\") "
Sep 10 16:30:37.783136 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.782976    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:30:37.783136 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.783089    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home" (OuterVolumeSpecName: "home") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:30:37.783317 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.783285    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:30:37.783461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.783317    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:37.783461 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.783362    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:37.785059 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.785026    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm" (OuterVolumeSpecName: "dshm") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:30:37.785185 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.785118    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d" (OuterVolumeSpecName: "kube-api-access-spl5d") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "kube-api-access-spl5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:30:37.785185 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.785129    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6ef3672-986f-4b65-b40d-8c4ae133d66c" (UID: "a6ef3672-986f-4b65-b40d-8c4ae133d66c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:37.884768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.884675    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6ef3672-986f-4b65-b40d-8c4ae133d66c-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:37.884768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.884707    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:37.884768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.884716    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6ef3672-986f-4b65-b40d-8c4ae133d66c-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:37.884768 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:37.884724    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spl5d\" (UniqueName: \"kubernetes.io/projected/a6ef3672-986f-4b65-b40d-8c4ae133d66c-kube-api-access-spl5d\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:38.038895 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:38.038854    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:30:38.045008 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:38.044977    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-219fd6/rate-test-model-219fd6-kserve-5fcf97f5c9-mpc8h"]
Sep 10 16:30:39.266427 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:39.266391    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" path="/var/lib/kubelet/pods/a6ef3672-986f-4b65-b40d-8c4ae133d66c/volumes"
Sep 10 16:30:41.850833 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:41.850796    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:30:41.851246 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:41.851008    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" podUID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" containerName="maas-api" containerID="cri-o://7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa" gracePeriod=30
Sep 10 16:30:42.102402 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.102342    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:30:42.125549 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.125506    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r26q6\" (UniqueName: \"kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6\") pod \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") "
Sep 10 16:30:42.125715 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.125625    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls\") pod \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") "
Sep 10 16:30:42.125715 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.125671    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs\") pod \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") "
Sep 10 16:30:42.125715 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.125704    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca\") pod \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\" (UID: \"ea48c037-05c7-4b9e-ac9a-0e06480482a9\") "
Sep 10 16:30:42.126387 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.126318    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea48c037-05c7-4b9e-ac9a-0e06480482a9" (UID: "ea48c037-05c7-4b9e-ac9a-0e06480482a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:30:42.128263 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.128219    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6" (OuterVolumeSpecName: "kube-api-access-r26q6") pod "ea48c037-05c7-4b9e-ac9a-0e06480482a9" (UID: "ea48c037-05c7-4b9e-ac9a-0e06480482a9"). InnerVolumeSpecName "kube-api-access-r26q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:30:42.129114 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.129060    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea48c037-05c7-4b9e-ac9a-0e06480482a9" (UID: "ea48c037-05c7-4b9e-ac9a-0e06480482a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:42.129260 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.129228    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea48c037-05c7-4b9e-ac9a-0e06480482a9" (UID: "ea48c037-05c7-4b9e-ac9a-0e06480482a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:42.227790 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.227749    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:42.227790 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.227781    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea48c037-05c7-4b9e-ac9a-0e06480482a9-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:42.227790 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.227792    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r26q6\" (UniqueName: \"kubernetes.io/projected/ea48c037-05c7-4b9e-ac9a-0e06480482a9-kube-api-access-r26q6\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:42.227790 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.227801    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea48c037-05c7-4b9e-ac9a-0e06480482a9-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:30:42.738801 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.738765    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" containerID="7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa" exitCode=0
Sep 10 16:30:42.738974 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.738825    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"
Sep 10 16:30:42.738974 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.738849    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" event={"ID":"ea48c037-05c7-4b9e-ac9a-0e06480482a9","Type":"ContainerDied","Data":"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"}
Sep 10 16:30:42.738974 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.738909    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9" event={"ID":"ea48c037-05c7-4b9e-ac9a-0e06480482a9","Type":"ContainerDied","Data":"abbc73a376f862a1db1ecac66924985ed47d6356e9914ced4a1030413e267439"}
Sep 10 16:30:42.738974 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.738935    2574 scope.go:117] "RemoveContainer" containerID="7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"
Sep 10 16:30:42.747970 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.747955    2574 scope.go:117] "RemoveContainer" containerID="7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"
Sep 10 16:30:42.748342 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:30:42.748297    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa\": container with ID starting with 7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa not found: ID does not exist" containerID="7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"
Sep 10 16:30:42.748415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.748357    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa"} err="failed to get container status \"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa\": rpc error: code = NotFound desc = could not find container \"7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa\": container with ID starting with 7e74c12d76fd81b5cb41541b7a8fd32df644226a1adfff5b359a03c0f76a36fa not found: ID does not exist"
Sep 10 16:30:42.763735 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.763712    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:30:42.769470 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:42.769442    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cde4a2-5c68697666-wlsq9"]
Sep 10 16:30:43.266226 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:30:43.266189    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" path="/var/lib/kubelet/pods/ea48c037-05c7-4b9e-ac9a-0e06480482a9/volumes"
Sep 10 16:31:01.345994 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.345950    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:31:01.346619 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.346242    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" podUID="bf9a6cf1-085d-4224-ac91-fe8665d18003" containerName="maas-api" containerID="cri-o://a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a" gracePeriod=30
Sep 10 16:31:01.593520 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.593491    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:31:01.704944 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.704840    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpw8f\" (UniqueName: \"kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f\") pod \"bf9a6cf1-085d-4224-ac91-fe8665d18003\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") "
Sep 10 16:31:01.704944 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.704902    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs\") pod \"bf9a6cf1-085d-4224-ac91-fe8665d18003\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") "
Sep 10 16:31:01.704944 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.704941    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca\") pod \"bf9a6cf1-085d-4224-ac91-fe8665d18003\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") "
Sep 10 16:31:01.705221 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.705013    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") pod \"bf9a6cf1-085d-4224-ac91-fe8665d18003\" (UID: \"bf9a6cf1-085d-4224-ac91-fe8665d18003\") "
Sep 10 16:31:01.705366 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.705322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf9a6cf1-085d-4224-ac91-fe8665d18003" (UID: "bf9a6cf1-085d-4224-ac91-fe8665d18003"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:31:01.707083 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.707058    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bf9a6cf1-085d-4224-ac91-fe8665d18003" (UID: "bf9a6cf1-085d-4224-ac91-fe8665d18003"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:01.707182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.707084    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f" (OuterVolumeSpecName: "kube-api-access-dpw8f") pod "bf9a6cf1-085d-4224-ac91-fe8665d18003" (UID: "bf9a6cf1-085d-4224-ac91-fe8665d18003"). InnerVolumeSpecName "kube-api-access-dpw8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:31:01.707182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.707088    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf9a6cf1-085d-4224-ac91-fe8665d18003" (UID: "bf9a6cf1-085d-4224-ac91-fe8665d18003"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:01.806421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.806384    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:01.806421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.806415    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpw8f\" (UniqueName: \"kubernetes.io/projected/bf9a6cf1-085d-4224-ac91-fe8665d18003-kube-api-access-dpw8f\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:01.806421 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.806425    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf9a6cf1-085d-4224-ac91-fe8665d18003-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:01.806703 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.806434    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf9a6cf1-085d-4224-ac91-fe8665d18003-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:01.829903 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.829866    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf9a6cf1-085d-4224-ac91-fe8665d18003" containerID="a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a" exitCode=0
Sep 10 16:31:01.830052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.829918    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"
Sep 10 16:31:01.830052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.829941    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" event={"ID":"bf9a6cf1-085d-4224-ac91-fe8665d18003","Type":"ContainerDied","Data":"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"}
Sep 10 16:31:01.830052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.829993    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b" event={"ID":"bf9a6cf1-085d-4224-ac91-fe8665d18003","Type":"ContainerDied","Data":"fc6fcc91fe4ec761dfaac7534b49e1ce2b23200eb74674f94abe9fd6287bbdee"}
Sep 10 16:31:01.830052 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.830015    2574 scope.go:117] "RemoveContainer" containerID="a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"
Sep 10 16:31:01.839069 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.839050    2574 scope.go:117] "RemoveContainer" containerID="a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"
Sep 10 16:31:01.839377 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:31:01.839356    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a\": container with ID starting with a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a not found: ID does not exist" containerID="a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"
Sep 10 16:31:01.839445 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.839387    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a"} err="failed to get container status \"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a\": rpc error: code = NotFound desc = could not find container \"a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a\": container with ID starting with a37990eae9296b903f1f84c9b174aa0250561826e60877fb019a910824ad3f0a not found: ID does not exist"
Sep 10 16:31:01.858664 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.858633    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:31:01.861196 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.861170    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-29fd7d-6b84879777-hx94b"]
Sep 10 16:31:01.982094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982017    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:31:01.982381 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982368    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" containerName="main"
Sep 10 16:31:01.982443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982382    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" containerName="main"
Sep 10 16:31:01.982443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982393    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" containerName="maas-api"
Sep 10 16:31:01.982443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982398    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" containerName="maas-api"
Sep 10 16:31:01.982443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982415    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf9a6cf1-085d-4224-ac91-fe8665d18003" containerName="maas-api"
Sep 10 16:31:01.982443 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982421    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf9a6cf1-085d-4224-ac91-fe8665d18003" containerName="maas-api"
Sep 10 16:31:01.982596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982482    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ef3672-986f-4b65-b40d-8c4ae133d66c" containerName="main"
Sep 10 16:31:01.982596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982490    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf9a6cf1-085d-4224-ac91-fe8665d18003" containerName="maas-api"
Sep 10 16:31:01.982596 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.982499    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea48c037-05c7-4b9e-ac9a-0e06480482a9" containerName="maas-api"
Sep 10 16:31:01.986959 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.986934    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:01.993251 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.993228    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-275a06-serving-cert\""
Sep 10 16:31:01.998618 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:01.998584    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:31:02.109037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.109002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.109037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.109041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.109261 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.109123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.109261 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.109182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t62js\" (UniqueName: \"kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.210095 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.210038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.210291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.210114    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.210291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.210149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.210291 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:31:02.210173    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-serving-cert: secret "maas-api-e2e-ipp-b-275a06-serving-cert" not found
Sep 10 16:31:02.210291 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.210206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t62js\" (UniqueName: \"kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.210291 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:31:02.210260    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls podName:ee184f5c-be76-4cee-aa4a-41ee47951664 nodeName:}" failed. No retries permitted until 2026-09-10 16:31:02.710242688 +0000 UTC m=+1757.943190772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls") pod "maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" (UID: "ee184f5c-be76-4cee-aa4a-41ee47951664") : secret "maas-api-e2e-ipp-b-275a06-serving-cert" not found
Sep 10 16:31:02.210783 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.210764    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.212627 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.212605    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.221846 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.221817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t62js\" (UniqueName: \"kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.716000 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.715968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.718287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.718267    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") pod \"maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:02.897910 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:02.897871    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:03.177819 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.177708    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:31:03.180312 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:31:03.180284    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee184f5c_be76_4cee_aa4a_41ee47951664.slice/crio-f24fec2150f9e0d711f6e2f583a135443bc7608c6e3fbfbdc25d26aa10979249 WatchSource:0}: Error finding container f24fec2150f9e0d711f6e2f583a135443bc7608c6e3fbfbdc25d26aa10979249: Status 404 returned error can't find the container with id f24fec2150f9e0d711f6e2f583a135443bc7608c6e3fbfbdc25d26aa10979249
Sep 10 16:31:03.266071 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.266036    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf9a6cf1-085d-4224-ac91-fe8665d18003" path="/var/lib/kubelet/pods/bf9a6cf1-085d-4224-ac91-fe8665d18003/volumes"
Sep 10 16:31:03.841729 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.841687    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" event={"ID":"ee184f5c-be76-4cee-aa4a-41ee47951664","Type":"ContainerStarted","Data":"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"}
Sep 10 16:31:03.842148 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.841760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" event={"ID":"ee184f5c-be76-4cee-aa4a-41ee47951664","Type":"ContainerStarted","Data":"f24fec2150f9e0d711f6e2f583a135443bc7608c6e3fbfbdc25d26aa10979249"}
Sep 10 16:31:03.842148 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.841897    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:03.866843 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:03.866794    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" podStartSLOduration=2.5801374299999997 podStartE2EDuration="2.866779825s" totalImagesPullingTime="286.642395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:31:03.86535092 +0000 UTC m=+1759.098299015" watchObservedRunningTime="2026-09-10 16:31:03.866779825 +0000 UTC m=+1759.099727927"
Sep 10 16:31:09.851800 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:09.851765    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:31:19.693427 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.693386    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:31:19.693852 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.693648    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" podUID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" containerName="maas-api" containerID="cri-o://af8f9f910bf0b82e1b50e70eee7988175259fa3486e322b0a75b275a349e1cf1" gracePeriod=30
Sep 10 16:31:19.915763 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.915729    2574 generic.go:356] "Generic (PLEG): container finished" podID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" containerID="af8f9f910bf0b82e1b50e70eee7988175259fa3486e322b0a75b275a349e1cf1" exitCode=0
Sep 10 16:31:19.915961 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.915804    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" event={"ID":"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0","Type":"ContainerDied","Data":"af8f9f910bf0b82e1b50e70eee7988175259fa3486e322b0a75b275a349e1cf1"}
Sep 10 16:31:19.942813 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.942786    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:31:19.985502 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.985417    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs\") pod \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") "
Sep 10 16:31:19.985502 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.985466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjqvh\" (UniqueName: \"kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh\") pod \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") "
Sep 10 16:31:19.985502 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.985496    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") pod \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") "
Sep 10 16:31:19.985773 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.985559    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca\") pod \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\" (UID: \"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0\") "
Sep 10 16:31:19.986057 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.986016    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" (UID: "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:31:19.987753 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.987719    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" (UID: "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:19.988162 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.988130    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" (UID: "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:19.988243 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:19.988144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh" (OuterVolumeSpecName: "kube-api-access-rjqvh") pod "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" (UID: "1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0"). InnerVolumeSpecName "kube-api-access-rjqvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:31:20.087132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.087096    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:20.087132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.087125    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjqvh\" (UniqueName: \"kubernetes.io/projected/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-kube-api-access-rjqvh\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:20.087132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.087135    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:20.087387 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.087145    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:31:20.922447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.922409    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm" event={"ID":"1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0","Type":"ContainerDied","Data":"32c3727ed1b64475769a99d7833994652c5d187d68071834569839331f425196"}
Sep 10 16:31:20.922447 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.922442    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"
Sep 10 16:31:20.922929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.922482    2574 scope.go:117] "RemoveContainer" containerID="af8f9f910bf0b82e1b50e70eee7988175259fa3486e322b0a75b275a349e1cf1"
Sep 10 16:31:20.948403 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.948373    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:31:20.954091 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:20.954066    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-90bd7e-888669675-wvvfm"]
Sep 10 16:31:21.266763 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:21.266681    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" path="/var/lib/kubelet/pods/1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0/volumes"
Sep 10 16:31:45.310612 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.310582    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:31:45.313003 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.312978    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:31:45.918994 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.918956    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:31:45.919303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.919290    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" containerName="maas-api"
Sep 10 16:31:45.919371 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.919304    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" containerName="maas-api"
Sep 10 16:31:45.919413 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.919391    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4e3bdf-1a3b-4b1a-8950-b5d0f1ffbbc0" containerName="maas-api"
Sep 10 16:31:45.923800 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.923783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:45.926860 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.926836    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-814d40\"/\"kube-root-ca.crt\""
Sep 10 16:31:45.928415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.928397    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-814d40\"/\"openshift-service-ca.crt\""
Sep 10 16:31:45.928415 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.928414    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-814d40\"/\"default-dockercfg-jz7cr\""
Sep 10 16:31:45.928583 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.928414    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-814d40\"/\"ipp-route-814d40-kserve-self-signed-certs\""
Sep 10 16:31:45.934860 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:45.934833    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:31:46.013513 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9q68\" (UniqueName: \"kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.013674 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.013674 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.013674 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013669    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.013786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.013786 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.013731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115391 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115570 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9q68\" (UniqueName: \"kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115570 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115684 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115626    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115690    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.115818 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.115796    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.117467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.117448    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.117734 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.117718    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.127010 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.126988    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9q68\" (UniqueName: \"kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68\") pod \"ipp-route-814d40-kserve-6564c98b8c-f5x8c\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") " pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.234195 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.234103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:46.507804 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:46.507711    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:31:46.510767 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:31:46.510741    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba6d8748_fc0e_44cb_b75e_14f4ea901185.slice/crio-92eb614c5a670c55c1e73eb0eb7012326cdf17dba9edde60dcdfac6e320032ab WatchSource:0}: Error finding container 92eb614c5a670c55c1e73eb0eb7012326cdf17dba9edde60dcdfac6e320032ab: Status 404 returned error can't find the container with id 92eb614c5a670c55c1e73eb0eb7012326cdf17dba9edde60dcdfac6e320032ab
Sep 10 16:31:47.043517 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:47.043478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" event={"ID":"ba6d8748-fc0e-44cb-b75e-14f4ea901185","Type":"ContainerStarted","Data":"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"}
Sep 10 16:31:47.043517 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:47.043524    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" event={"ID":"ba6d8748-fc0e-44cb-b75e-14f4ea901185","Type":"ContainerStarted","Data":"92eb614c5a670c55c1e73eb0eb7012326cdf17dba9edde60dcdfac6e320032ab"}
Sep 10 16:31:47.075951 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:47.075890    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" podStartSLOduration=2.075872043 podStartE2EDuration="2.075872043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:31:47.073723794 +0000 UTC m=+1802.306671899" watchObservedRunningTime="2026-09-10 16:31:47.075872043 +0000 UTC m=+1802.308820145"
Sep 10 16:31:56.234439 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:56.234396    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:56.234836 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:56.234466    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:56.247774 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:56.247736    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:31:57.101803 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:31:57.101772    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:32:17.517684 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.517651    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:17.521134 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.521114    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.524609 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.524586    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-68ca72-serving-cert\""
Sep 10 16:32:17.539613 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.539584    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:17.590753 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.590721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.590991 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.590769    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.590991 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.590879    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.590991 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.590921    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9s7d\" (UniqueName: \"kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.692213 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.692178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.692213 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.692226    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.692523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.692280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.692523 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.692302    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9s7d\" (UniqueName: \"kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.692523 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:32:17.692446    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-serving-cert: secret "maas-api-e2e-ipp-cleanup-68ca72-serving-cert" not found
Sep 10 16:32:17.692693 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:32:17.692530    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls podName:65671321-9b87-43d8-891a-a42f5f087deb nodeName:}" failed. No retries permitted until 2026-09-10 16:32:18.192509732 +0000 UTC m=+1833.425457818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" (UID: "65671321-9b87-43d8-891a-a42f5f087deb") : secret "maas-api-e2e-ipp-cleanup-68ca72-serving-cert" not found
Sep 10 16:32:17.692992 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.692960    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.694772 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.694753    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:17.701737 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:17.701715    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9s7d\" (UniqueName: \"kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:18.197533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:18.197499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:18.197702 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:32:18.197634    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-serving-cert: secret "maas-api-e2e-ipp-cleanup-68ca72-serving-cert" not found
Sep 10 16:32:18.197747 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:32:18.197704    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls podName:65671321-9b87-43d8-891a-a42f5f087deb nodeName:}" failed. No retries permitted until 2026-09-10 16:32:19.19768828 +0000 UTC m=+1834.430636360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" (UID: "65671321-9b87-43d8-891a-a42f5f087deb") : secret "maas-api-e2e-ipp-cleanup-68ca72-serving-cert" not found
Sep 10 16:32:19.206923 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:19.206884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:19.209524 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:19.209502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:19.331355 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:19.331299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:19.605424 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:19.605395    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:19.607235 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:32:19.607195    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65671321_9b87_43d8_891a_a42f5f087deb.slice/crio-2e24c71c41d7276ef753384be82a9248277f053c52464981a577c7cea6caed4a WatchSource:0}: Error finding container 2e24c71c41d7276ef753384be82a9248277f053c52464981a577c7cea6caed4a: Status 404 returned error can't find the container with id 2e24c71c41d7276ef753384be82a9248277f053c52464981a577c7cea6caed4a
Sep 10 16:32:20.199403 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:20.199364    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" event={"ID":"65671321-9b87-43d8-891a-a42f5f087deb","Type":"ContainerStarted","Data":"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"}
Sep 10 16:32:20.199587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:20.199419    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" event={"ID":"65671321-9b87-43d8-891a-a42f5f087deb","Type":"ContainerStarted","Data":"2e24c71c41d7276ef753384be82a9248277f053c52464981a577c7cea6caed4a"}
Sep 10 16:32:20.199587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:20.199565    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:20.222598 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:20.222547    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" podStartSLOduration=2.758392555 podStartE2EDuration="3.222530762s" totalImagesPullingTime="464.138207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:32:20.22157139 +0000 UTC m=+1835.454519492" watchObservedRunningTime="2026-09-10 16:32:20.222530762 +0000 UTC m=+1835.455478891"
Sep 10 16:32:26.209011 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:26.208982    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:48.049787 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.049748    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:48.050203 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.050003    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" podUID="65671321-9b87-43d8-891a-a42f5f087deb" containerName="maas-api" containerID="cri-o://68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815" gracePeriod=30
Sep 10 16:32:48.299988 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.299915    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:48.334196 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.334159    2574 generic.go:356] "Generic (PLEG): container finished" podID="65671321-9b87-43d8-891a-a42f5f087deb" containerID="68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815" exitCode=0
Sep 10 16:32:48.334408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.334226    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" event={"ID":"65671321-9b87-43d8-891a-a42f5f087deb","Type":"ContainerDied","Data":"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"}
Sep 10 16:32:48.334408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.334234    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"
Sep 10 16:32:48.334408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.334275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc" event={"ID":"65671321-9b87-43d8-891a-a42f5f087deb","Type":"ContainerDied","Data":"2e24c71c41d7276ef753384be82a9248277f053c52464981a577c7cea6caed4a"}
Sep 10 16:32:48.334408 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.334309    2574 scope.go:117] "RemoveContainer" containerID="68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"
Sep 10 16:32:48.343056 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.343034    2574 scope.go:117] "RemoveContainer" containerID="68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"
Sep 10 16:32:48.343316 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:32:48.343295    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815\": container with ID starting with 68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815 not found: ID does not exist" containerID="68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"
Sep 10 16:32:48.343388 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.343338    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815"} err="failed to get container status \"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815\": rpc error: code = NotFound desc = could not find container \"68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815\": container with ID starting with 68416c1fefd2469071614a4b867564d3b7b86dfe61c914ebf533685777f82815 not found: ID does not exist"
Sep 10 16:32:48.363871 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.363845    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9s7d\" (UniqueName: \"kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d\") pod \"65671321-9b87-43d8-891a-a42f5f087deb\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") "
Sep 10 16:32:48.363982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.363908    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") pod \"65671321-9b87-43d8-891a-a42f5f087deb\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") "
Sep 10 16:32:48.363982 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.363934    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs\") pod \"65671321-9b87-43d8-891a-a42f5f087deb\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") "
Sep 10 16:32:48.364092 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.363992    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca\") pod \"65671321-9b87-43d8-891a-a42f5f087deb\" (UID: \"65671321-9b87-43d8-891a-a42f5f087deb\") "
Sep 10 16:32:48.364384 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.364356    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "65671321-9b87-43d8-891a-a42f5f087deb" (UID: "65671321-9b87-43d8-891a-a42f5f087deb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:32:48.366021 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.366000    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "65671321-9b87-43d8-891a-a42f5f087deb" (UID: "65671321-9b87-43d8-891a-a42f5f087deb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:32:48.366132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.366019    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "65671321-9b87-43d8-891a-a42f5f087deb" (UID: "65671321-9b87-43d8-891a-a42f5f087deb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:32:48.366132 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.366103    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d" (OuterVolumeSpecName: "kube-api-access-m9s7d") pod "65671321-9b87-43d8-891a-a42f5f087deb" (UID: "65671321-9b87-43d8-891a-a42f5f087deb"). InnerVolumeSpecName "kube-api-access-m9s7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:32:48.465467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.465439    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9s7d\" (UniqueName: \"kubernetes.io/projected/65671321-9b87-43d8-891a-a42f5f087deb-kube-api-access-m9s7d\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:32:48.465467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.465466    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:32:48.465467 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.465476    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65671321-9b87-43d8-891a-a42f5f087deb-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:32:48.465680 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.465484    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/65671321-9b87-43d8-891a-a42f5f087deb-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:32:48.659198 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.659165    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:48.665914 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:48.665888    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-68ca72-5dfd67f889-qcqzc"]
Sep 10 16:32:49.266594 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:32:49.266558    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65671321-9b87-43d8-891a-a42f5f087deb" path="/var/lib/kubelet/pods/65671321-9b87-43d8-891a-a42f5f087deb/volumes"
Sep 10 16:33:14.916835 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:14.916748    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:33:14.917287 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:14.917062    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" podUID="a8ed8e54-d109-4da7-81f4-a871af667316" containerName="maas-api" containerID="cri-o://f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a" gracePeriod=30
Sep 10 16:33:15.165933 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.165907    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:33:15.313912 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.313878    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls\") pod \"a8ed8e54-d109-4da7-81f4-a871af667316\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") "
Sep 10 16:33:15.314093 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.313934    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdkcr\" (UniqueName: \"kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr\") pod \"a8ed8e54-d109-4da7-81f4-a871af667316\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") "
Sep 10 16:33:15.314093 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.313964    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca\") pod \"a8ed8e54-d109-4da7-81f4-a871af667316\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") "
Sep 10 16:33:15.314093 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.314085    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs\") pod \"a8ed8e54-d109-4da7-81f4-a871af667316\" (UID: \"a8ed8e54-d109-4da7-81f4-a871af667316\") "
Sep 10 16:33:15.314342 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.314300    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8ed8e54-d109-4da7-81f4-a871af667316" (UID: "a8ed8e54-d109-4da7-81f4-a871af667316"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:33:15.314524 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.314498    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8ed8e54-d109-4da7-81f4-a871af667316-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:15.316392 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.316361    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8ed8e54-d109-4da7-81f4-a871af667316" (UID: "a8ed8e54-d109-4da7-81f4-a871af667316"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:15.316692 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.316662    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8ed8e54-d109-4da7-81f4-a871af667316" (UID: "a8ed8e54-d109-4da7-81f4-a871af667316"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:15.316789 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.316762    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr" (OuterVolumeSpecName: "kube-api-access-jdkcr") pod "a8ed8e54-d109-4da7-81f4-a871af667316" (UID: "a8ed8e54-d109-4da7-81f4-a871af667316"). InnerVolumeSpecName "kube-api-access-jdkcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:33:15.415450 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.415406    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:15.415450 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.415438    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8ed8e54-d109-4da7-81f4-a871af667316-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:15.415450 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.415447    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdkcr\" (UniqueName: \"kubernetes.io/projected/a8ed8e54-d109-4da7-81f4-a871af667316-kube-api-access-jdkcr\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:15.464929 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.464899    2574 generic.go:356] "Generic (PLEG): container finished" podID="a8ed8e54-d109-4da7-81f4-a871af667316" containerID="f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a" exitCode=0
Sep 10 16:33:15.465096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.464943    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"
Sep 10 16:33:15.465096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.464987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" event={"ID":"a8ed8e54-d109-4da7-81f4-a871af667316","Type":"ContainerDied","Data":"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"}
Sep 10 16:33:15.465096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.465049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv" event={"ID":"a8ed8e54-d109-4da7-81f4-a871af667316","Type":"ContainerDied","Data":"7faa7987463874fe994c27faf23a5f78f93241f090afef4f5d758373b6f6ee70"}
Sep 10 16:33:15.465096 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.465074    2574 scope.go:117] "RemoveContainer" containerID="f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"
Sep 10 16:33:15.474679 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.474664    2574 scope.go:117] "RemoveContainer" containerID="f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"
Sep 10 16:33:15.474982 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:33:15.474953    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a\": container with ID starting with f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a not found: ID does not exist" containerID="f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"
Sep 10 16:33:15.475077 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.474988    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a"} err="failed to get container status \"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a\": rpc error: code = NotFound desc = could not find container \"f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a\": container with ID starting with f05dfd817cec07e0b7fb7aca51bd8ec670dd46eb72bd1a8049fb3584d5a5109a not found: ID does not exist"
Sep 10 16:33:15.492922 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.492896    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:33:15.497873 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:15.497853    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-814d40-66c7dff7d-phsvv"]
Sep 10 16:33:17.266861 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:17.266823    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8ed8e54-d109-4da7-81f4-a871af667316" path="/var/lib/kubelet/pods/a8ed8e54-d109-4da7-81f4-a871af667316/volumes"
Sep 10 16:33:20.550764 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:20.550730    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:33:35.570311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.570252    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" podUID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" containerName="main" containerID="cri-o://d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7" gracePeriod=45
Sep 10 16:33:35.812384 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.812361    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:33:35.890397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890302    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890377    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890439    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9q68\" (UniqueName: \"kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890479    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890506    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890523    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm\") pod \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\" (UID: \"ba6d8748-fc0e-44cb-b75e-14f4ea901185\") "
Sep 10 16:33:35.890626 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890567    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:33:35.890897 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890680    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home" (OuterVolumeSpecName: "home") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:33:35.890897 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890768    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:33:35.890897 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890832    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-model-cache\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:35.890897 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.890851    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-home\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:35.892828 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.892802    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68" (OuterVolumeSpecName: "kube-api-access-g9q68") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "kube-api-access-g9q68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:33:35.892909 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.892867    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm" (OuterVolumeSpecName: "dshm") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:33:35.892947 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.892898    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba6d8748-fc0e-44cb-b75e-14f4ea901185" (UID: "ba6d8748-fc0e-44cb-b75e-14f4ea901185"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:35.991689 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.991662    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9q68\" (UniqueName: \"kubernetes.io/projected/ba6d8748-fc0e-44cb-b75e-14f4ea901185-kube-api-access-g9q68\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:35.991689 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.991687    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tmp-dir\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:35.991851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.991698    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6d8748-fc0e-44cb-b75e-14f4ea901185-tls-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:35.991851 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:35.991708    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6d8748-fc0e-44cb-b75e-14f4ea901185-dshm\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:33:36.560581 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.560545    2574 generic.go:356] "Generic (PLEG): container finished" podID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" containerID="d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7" exitCode=0
Sep 10 16:33:36.560744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.560581    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"
Sep 10 16:33:36.560744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.560629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" event={"ID":"ba6d8748-fc0e-44cb-b75e-14f4ea901185","Type":"ContainerDied","Data":"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"}
Sep 10 16:33:36.560744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.560670    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c" event={"ID":"ba6d8748-fc0e-44cb-b75e-14f4ea901185","Type":"ContainerDied","Data":"92eb614c5a670c55c1e73eb0eb7012326cdf17dba9edde60dcdfac6e320032ab"}
Sep 10 16:33:36.560744 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.560689    2574 scope.go:117] "RemoveContainer" containerID="d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"
Sep 10 16:33:36.569663 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.569647    2574 scope.go:117] "RemoveContainer" containerID="d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"
Sep 10 16:33:36.569913 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:33:36.569893    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7\": container with ID starting with d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7 not found: ID does not exist" containerID="d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"
Sep 10 16:33:36.569957 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.569922    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7"} err="failed to get container status \"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7\": rpc error: code = NotFound desc = could not find container \"d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7\": container with ID starting with d8b300ff78cbab21f9aef6cf7608792a0a005288821027cc86bf9c7e585743d7 not found: ID does not exist"
Sep 10 16:33:36.583184 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.583162    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:33:36.589246 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:36.589227    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-814d40/ipp-route-814d40-kserve-6564c98b8c-f5x8c"]
Sep 10 16:33:37.265674 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:33:37.265640    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" path="/var/lib/kubelet/pods/ba6d8748-fc0e-44cb-b75e-14f4ea901185/volumes"
Sep 10 16:34:15.919007 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:15.918966    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:34:15.919625 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:15.919279    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" podUID="ee184f5c-be76-4cee-aa4a-41ee47951664" containerName="maas-api" containerID="cri-o://f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9" gracePeriod=30
Sep 10 16:34:16.162819 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.162790    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:34:16.247434 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.247348    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t62js\" (UniqueName: \"kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js\") pod \"ee184f5c-be76-4cee-aa4a-41ee47951664\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") "
Sep 10 16:34:16.247434 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.247398    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs\") pod \"ee184f5c-be76-4cee-aa4a-41ee47951664\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") "
Sep 10 16:34:16.247434 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.247423    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca\") pod \"ee184f5c-be76-4cee-aa4a-41ee47951664\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") "
Sep 10 16:34:16.247723 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.247443    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") pod \"ee184f5c-be76-4cee-aa4a-41ee47951664\" (UID: \"ee184f5c-be76-4cee-aa4a-41ee47951664\") "
Sep 10 16:34:16.247841 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.247816    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee184f5c-be76-4cee-aa4a-41ee47951664" (UID: "ee184f5c-be76-4cee-aa4a-41ee47951664"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:16.249709 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.249673    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee184f5c-be76-4cee-aa4a-41ee47951664" (UID: "ee184f5c-be76-4cee-aa4a-41ee47951664"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:16.249709 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.249694    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee184f5c-be76-4cee-aa4a-41ee47951664" (UID: "ee184f5c-be76-4cee-aa4a-41ee47951664"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:16.249874 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.249706    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js" (OuterVolumeSpecName: "kube-api-access-t62js") pod "ee184f5c-be76-4cee-aa4a-41ee47951664" (UID: "ee184f5c-be76-4cee-aa4a-41ee47951664"). InnerVolumeSpecName "kube-api-access-t62js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:16.349040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.349001    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t62js\" (UniqueName: \"kubernetes.io/projected/ee184f5c-be76-4cee-aa4a-41ee47951664-kube-api-access-t62js\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:16.349040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.349032    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:16.349040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.349042    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee184f5c-be76-4cee-aa4a-41ee47951664-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:16.349040 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.349051    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee184f5c-be76-4cee-aa4a-41ee47951664-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:16.743236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.743199    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee184f5c-be76-4cee-aa4a-41ee47951664" containerID="f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9" exitCode=0
Sep 10 16:34:16.743462 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.743243    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"
Sep 10 16:34:16.743462 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.743287    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" event={"ID":"ee184f5c-be76-4cee-aa4a-41ee47951664","Type":"ContainerDied","Data":"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"}
Sep 10 16:34:16.743462 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.743365    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj" event={"ID":"ee184f5c-be76-4cee-aa4a-41ee47951664","Type":"ContainerDied","Data":"f24fec2150f9e0d711f6e2f583a135443bc7608c6e3fbfbdc25d26aa10979249"}
Sep 10 16:34:16.743462 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.743397    2574 scope.go:117] "RemoveContainer" containerID="f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"
Sep 10 16:34:16.752622 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.752602    2574 scope.go:117] "RemoveContainer" containerID="f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"
Sep 10 16:34:16.752901 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:34:16.752885    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9\": container with ID starting with f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9 not found: ID does not exist" containerID="f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"
Sep 10 16:34:16.752956 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.752909    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9"} err="failed to get container status \"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9\": rpc error: code = NotFound desc = could not find container \"f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9\": container with ID starting with f00eaf7f826caad04a4d8b57810df839a8e2e95d521a0007f005369367c923b9 not found: ID does not exist"
Sep 10 16:34:16.770624 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.770594    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:34:16.775541 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:16.775515    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-275a06-84bf5bf975-grnxj"]
Sep 10 16:34:17.267639 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:17.267597    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee184f5c-be76-4cee-aa4a-41ee47951664" path="/var/lib/kubelet/pods/ee184f5c-be76-4cee-aa4a-41ee47951664/volumes"
Sep 10 16:34:33.829970 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:33.829934    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:34:33.830565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:33.830142    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" podUID="686b2132-cc10-4bfd-a716-d05818bfa12f" containerName="maas-api" containerID="cri-o://3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9" gracePeriod=30
Sep 10 16:34:34.078608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.078582    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:34:34.202695 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.202605    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") pod \"686b2132-cc10-4bfd-a716-d05818bfa12f\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") "
Sep 10 16:34:34.202850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.202698    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca\") pod \"686b2132-cc10-4bfd-a716-d05818bfa12f\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") "
Sep 10 16:34:34.202850 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.202802    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzhnt\" (UniqueName: \"kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt\") pod \"686b2132-cc10-4bfd-a716-d05818bfa12f\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") "
Sep 10 16:34:34.202939 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.202858    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs\") pod \"686b2132-cc10-4bfd-a716-d05818bfa12f\" (UID: \"686b2132-cc10-4bfd-a716-d05818bfa12f\") "
Sep 10 16:34:34.203553 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.203507    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "686b2132-cc10-4bfd-a716-d05818bfa12f" (UID: "686b2132-cc10-4bfd-a716-d05818bfa12f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:34.206537 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.203939    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/686b2132-cc10-4bfd-a716-d05818bfa12f-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:34.206537 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.205646    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt" (OuterVolumeSpecName: "kube-api-access-fzhnt") pod "686b2132-cc10-4bfd-a716-d05818bfa12f" (UID: "686b2132-cc10-4bfd-a716-d05818bfa12f"). InnerVolumeSpecName "kube-api-access-fzhnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:34.211125 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.211095    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "686b2132-cc10-4bfd-a716-d05818bfa12f" (UID: "686b2132-cc10-4bfd-a716-d05818bfa12f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:34.211416 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.211318    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "686b2132-cc10-4bfd-a716-d05818bfa12f" (UID: "686b2132-cc10-4bfd-a716-d05818bfa12f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:34.305073 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.305031    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzhnt\" (UniqueName: \"kubernetes.io/projected/686b2132-cc10-4bfd-a716-d05818bfa12f-kube-api-access-fzhnt\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:34.305073 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.305066    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:34.305073 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.305077    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/686b2132-cc10-4bfd-a716-d05818bfa12f-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:34.827952 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.827911    2574 generic.go:356] "Generic (PLEG): container finished" podID="686b2132-cc10-4bfd-a716-d05818bfa12f" containerID="3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9" exitCode=0
Sep 10 16:34:34.828133 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.827967    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" event={"ID":"686b2132-cc10-4bfd-a716-d05818bfa12f","Type":"ContainerDied","Data":"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"}
Sep 10 16:34:34.828133 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.827976    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"
Sep 10 16:34:34.828133 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.828023    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn" event={"ID":"686b2132-cc10-4bfd-a716-d05818bfa12f","Type":"ContainerDied","Data":"fbeca256a56501d634d145d3845983e57d8274cd033ca916fc983da8b82f96c7"}
Sep 10 16:34:34.828133 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.828053    2574 scope.go:117] "RemoveContainer" containerID="3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"
Sep 10 16:34:34.837183 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.836980    2574 scope.go:117] "RemoveContainer" containerID="3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"
Sep 10 16:34:34.837479 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:34:34.837278    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9\": container with ID starting with 3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9 not found: ID does not exist" containerID="3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"
Sep 10 16:34:34.837479 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.837306    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9"} err="failed to get container status \"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9\": rpc error: code = NotFound desc = could not find container \"3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9\": container with ID starting with 3d0af7e6047304403929c2de638abe835c151e5b82dfa95d72a4f83881c13cc9 not found: ID does not exist"
Sep 10 16:34:34.850102 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.850064    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:34:34.854136 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:34.854109    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-219fd6-6f7858c8c8-6hckn"]
Sep 10 16:34:35.266092 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:35.266005    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686b2132-cc10-4bfd-a716-d05818bfa12f" path="/var/lib/kubelet/pods/686b2132-cc10-4bfd-a716-d05818bfa12f/volumes"
Sep 10 16:34:52.603543 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.603509    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:34:52.604065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.603772    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" podUID="5b5475e8-d397-44de-8b1a-115982dba750" containerName="maas-api" containerID="cri-o://dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4" gracePeriod=30
Sep 10 16:34:52.843912 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.843886    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:34:52.871352 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.871251    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls\") pod \"5b5475e8-d397-44de-8b1a-115982dba750\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") "
Sep 10 16:34:52.871352 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.871293    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgv5h\" (UniqueName: \"kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h\") pod \"5b5475e8-d397-44de-8b1a-115982dba750\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") "
Sep 10 16:34:52.871555 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.871363    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs\") pod \"5b5475e8-d397-44de-8b1a-115982dba750\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") "
Sep 10 16:34:52.871555 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.871403    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca\") pod \"5b5475e8-d397-44de-8b1a-115982dba750\" (UID: \"5b5475e8-d397-44de-8b1a-115982dba750\") "
Sep 10 16:34:52.871827 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.871801    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b5475e8-d397-44de-8b1a-115982dba750" (UID: "5b5475e8-d397-44de-8b1a-115982dba750"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:52.873433 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.873403    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b5475e8-d397-44de-8b1a-115982dba750" (UID: "5b5475e8-d397-44de-8b1a-115982dba750"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:52.873540 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.873448    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h" (OuterVolumeSpecName: "kube-api-access-kgv5h") pod "5b5475e8-d397-44de-8b1a-115982dba750" (UID: "5b5475e8-d397-44de-8b1a-115982dba750"). InnerVolumeSpecName "kube-api-access-kgv5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:52.873540 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.873459    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b5475e8-d397-44de-8b1a-115982dba750" (UID: "5b5475e8-d397-44de-8b1a-115982dba750"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:52.912896 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.912865    2574 generic.go:356] "Generic (PLEG): container finished" podID="5b5475e8-d397-44de-8b1a-115982dba750" containerID="dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4" exitCode=0
Sep 10 16:34:52.913037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.912913    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"
Sep 10 16:34:52.913037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.912946    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" event={"ID":"5b5475e8-d397-44de-8b1a-115982dba750","Type":"ContainerDied","Data":"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"}
Sep 10 16:34:52.913037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.913006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx" event={"ID":"5b5475e8-d397-44de-8b1a-115982dba750","Type":"ContainerDied","Data":"983e9dba1fc1361366a7fabacb51c878a5a5b0e72c01e50f38a106c81c4a6860"}
Sep 10 16:34:52.913037 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.913027    2574 scope.go:117] "RemoveContainer" containerID="dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"
Sep 10 16:34:52.921961 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.921946    2574 scope.go:117] "RemoveContainer" containerID="dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"
Sep 10 16:34:52.922218 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:34:52.922202    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4\": container with ID starting with dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4 not found: ID does not exist" containerID="dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"
Sep 10 16:34:52.922262 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.922228    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4"} err="failed to get container status \"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4\": rpc error: code = NotFound desc = could not find container \"dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4\": container with ID starting with dff6f4ebd920034066bb33d79d433037f95c944d7e3cf23aa9eb80396902e8e4 not found: ID does not exist"
Sep 10 16:34:52.937285 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.937258    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:34:52.943491 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.943465    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b73303-f6d75cdcc-prbmx"]
Sep 10 16:34:52.972060 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.972041    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b5475e8-d397-44de-8b1a-115982dba750-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:52.972060 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.972061    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:52.972188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.972070    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgv5h\" (UniqueName: \"kubernetes.io/projected/5b5475e8-d397-44de-8b1a-115982dba750-kube-api-access-kgv5h\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:52.972188 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:52.972078    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b5475e8-d397-44de-8b1a-115982dba750-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:34:53.265605 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:34:53.265515    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5475e8-d397-44de-8b1a-115982dba750" path="/var/lib/kubelet/pods/5b5475e8-d397-44de-8b1a-115982dba750/volumes"
Sep 10 16:35:03.023271 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.023227    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:35:03.023734 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.023546    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" podUID="33e208a8-cfe7-4bee-8092-e35c2847240f" containerName="manager" containerID="cri-o://a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156" gracePeriod=30
Sep 10 16:35:03.270428 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.270401    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:35:03.357984 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.357945    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc4hk\" (UniqueName: \"kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk\") pod \"33e208a8-cfe7-4bee-8092-e35c2847240f\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") "
Sep 10 16:35:03.358153 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.358035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs\") pod \"33e208a8-cfe7-4bee-8092-e35c2847240f\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") "
Sep 10 16:35:03.358153 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.358075    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert\") pod \"33e208a8-cfe7-4bee-8092-e35c2847240f\" (UID: \"33e208a8-cfe7-4bee-8092-e35c2847240f\") "
Sep 10 16:35:03.359961 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.359928    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk" (OuterVolumeSpecName: "kube-api-access-zc4hk") pod "33e208a8-cfe7-4bee-8092-e35c2847240f" (UID: "33e208a8-cfe7-4bee-8092-e35c2847240f"). InnerVolumeSpecName "kube-api-access-zc4hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:35:03.360094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.360058    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "33e208a8-cfe7-4bee-8092-e35c2847240f" (UID: "33e208a8-cfe7-4bee-8092-e35c2847240f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:35:03.360094 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.360079    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33e208a8-cfe7-4bee-8092-e35c2847240f" (UID: "33e208a8-cfe7-4bee-8092-e35c2847240f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:35:03.459067 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.459034    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:35:03.459067 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.459060    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33e208a8-cfe7-4bee-8092-e35c2847240f-webhook-cert\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:35:03.459067 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.459069    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc4hk\" (UniqueName: \"kubernetes.io/projected/33e208a8-cfe7-4bee-8092-e35c2847240f-kube-api-access-zc4hk\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:35:03.962816 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.962780    2574 generic.go:356] "Generic (PLEG): container finished" podID="33e208a8-cfe7-4bee-8092-e35c2847240f" containerID="a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156" exitCode=0
Sep 10 16:35:03.962986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.962834    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-wx4kn"
Sep 10 16:35:03.962986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.962862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" event={"ID":"33e208a8-cfe7-4bee-8092-e35c2847240f","Type":"ContainerDied","Data":"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"}
Sep 10 16:35:03.962986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.962914    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-wx4kn" event={"ID":"33e208a8-cfe7-4bee-8092-e35c2847240f","Type":"ContainerDied","Data":"3ec1bcf8d203a13994861ff31e974bb3f05bc95d432f22631336407c2ac304e2"}
Sep 10 16:35:03.962986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.962935    2574 scope.go:117] "RemoveContainer" containerID="a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"
Sep 10 16:35:03.972144 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.972128    2574 scope.go:117] "RemoveContainer" containerID="a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"
Sep 10 16:35:03.972436 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:35:03.972417    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156\": container with ID starting with a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156 not found: ID does not exist" containerID="a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"
Sep 10 16:35:03.972508 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.972448    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156"} err="failed to get container status \"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156\": rpc error: code = NotFound desc = could not find container \"a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156\": container with ID starting with a304cb223e3d20462181976365a36be7fb9a7995881e0ba9fd6a0110531e3156 not found: ID does not exist"
Sep 10 16:35:03.985522 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.985499    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:35:03.990991 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:03.990969    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-wx4kn"]
Sep 10 16:35:05.265856 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:35:05.265823    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e208a8-cfe7-4bee-8092-e35c2847240f" path="/var/lib/kubelet/pods/33e208a8-cfe7-4bee-8092-e35c2847240f/volumes"
Sep 10 16:36:04.877839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.877796    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878254    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8ed8e54-d109-4da7-81f4-a871af667316" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878275    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8ed8e54-d109-4da7-81f4-a871af667316" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878300    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686b2132-cc10-4bfd-a716-d05818bfa12f" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878309    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="686b2132-cc10-4bfd-a716-d05818bfa12f" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878320    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee184f5c-be76-4cee-aa4a-41ee47951664" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878346    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee184f5c-be76-4cee-aa4a-41ee47951664" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878357    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" containerName="main"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878365    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" containerName="main"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878377    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65671321-9b87-43d8-891a-a42f5f087deb" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878385    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="65671321-9b87-43d8-891a-a42f5f087deb" containerName="maas-api"
Sep 10 16:36:04.878397 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878398    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5475e8-d397-44de-8b1a-115982dba750" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878406    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5475e8-d397-44de-8b1a-115982dba750" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878426    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e208a8-cfe7-4bee-8092-e35c2847240f" containerName="manager"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878434    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e208a8-cfe7-4bee-8092-e35c2847240f" containerName="manager"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878526    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5475e8-d397-44de-8b1a-115982dba750" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878540    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="65671321-9b87-43d8-891a-a42f5f087deb" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878551    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e208a8-cfe7-4bee-8092-e35c2847240f" containerName="manager"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878560    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8ed8e54-d109-4da7-81f4-a871af667316" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878571    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba6d8748-fc0e-44cb-b75e-14f4ea901185" containerName="main"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878581    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee184f5c-be76-4cee-aa4a-41ee47951664" containerName="maas-api"
Sep 10 16:36:04.878953 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.878591    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="686b2132-cc10-4bfd-a716-d05818bfa12f" containerName="maas-api"
Sep 10 16:36:04.881842 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.881822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:04.886154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.886124    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:36:04.886154 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.886139    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-phs7m\""
Sep 10 16:36:04.886311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.886145    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:36:04.886311 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.886193    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:36:04.894366 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.894341    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:04.971046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.971014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:04.971210 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.971065    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4dd2\" (UniqueName: \"kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:04.971210 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:04.971107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.072271 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.072233    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4dd2\" (UniqueName: \"kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.072271 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.072279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.072489 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.072316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.074790 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.074756    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.074909 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.074794    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.093986 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.093960    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4dd2\" (UniqueName: \"kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2\") pod \"maas-controller-ccf7f868f-6mr46\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") " pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.191527 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.191437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:05.465857 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.465812    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:05.468149 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:36:05.468125    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1cd4bce_415e_4dc5_b31d_076678f97295.slice/crio-ca27f5c391aa8501cb2b519026c6002949f1c515a77cc9099aab2c7beca4058a WatchSource:0}: Error finding container ca27f5c391aa8501cb2b519026c6002949f1c515a77cc9099aab2c7beca4058a: Status 404 returned error can't find the container with id ca27f5c391aa8501cb2b519026c6002949f1c515a77cc9099aab2c7beca4058a
Sep 10 16:36:05.469548 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:05.469529    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:36:06.237830 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:06.237797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-6mr46" event={"ID":"e1cd4bce-415e-4dc5-b31d-076678f97295","Type":"ContainerStarted","Data":"ca27f5c391aa8501cb2b519026c6002949f1c515a77cc9099aab2c7beca4058a"}
Sep 10 16:36:07.243736 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:07.243699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-6mr46" event={"ID":"e1cd4bce-415e-4dc5-b31d-076678f97295","Type":"ContainerStarted","Data":"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"}
Sep 10 16:36:07.244142 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:07.243826    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:07.269275 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:07.269223    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-ccf7f868f-6mr46" podStartSLOduration=2.485678405 podStartE2EDuration="3.269208759s" totalImagesPullingTime="783.530354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:36:07.266989093 +0000 UTC m=+2062.499937194" watchObservedRunningTime="2026-09-10 16:36:07.269208759 +0000 UTC m=+2062.502156860"
Sep 10 16:36:18.251265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:18.251184    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:25.573265 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.573226    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:25.573713 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.573467    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-ccf7f868f-6mr46" podUID="e1cd4bce-415e-4dc5-b31d-076678f97295" containerName="manager" containerID="cri-o://6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362" gracePeriod=30
Sep 10 16:36:25.824019 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.823952    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:25.957314 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.957272    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs\") pod \"e1cd4bce-415e-4dc5-b31d-076678f97295\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") "
Sep 10 16:36:25.957533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.957437    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert\") pod \"e1cd4bce-415e-4dc5-b31d-076678f97295\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") "
Sep 10 16:36:25.957533 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.957476    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4dd2\" (UniqueName: \"kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2\") pod \"e1cd4bce-415e-4dc5-b31d-076678f97295\" (UID: \"e1cd4bce-415e-4dc5-b31d-076678f97295\") "
Sep 10 16:36:25.959362 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.959310    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e1cd4bce-415e-4dc5-b31d-076678f97295" (UID: "e1cd4bce-415e-4dc5-b31d-076678f97295"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:36:25.959362 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.959348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e1cd4bce-415e-4dc5-b31d-076678f97295" (UID: "e1cd4bce-415e-4dc5-b31d-076678f97295"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:36:25.959514 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:25.959390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2" (OuterVolumeSpecName: "kube-api-access-g4dd2") pod "e1cd4bce-415e-4dc5-b31d-076678f97295" (UID: "e1cd4bce-415e-4dc5-b31d-076678f97295"). InnerVolumeSpecName "kube-api-access-g4dd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:36:26.058487 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.058447    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-webhook-cert\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:36:26.058487 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.058477    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4dd2\" (UniqueName: \"kubernetes.io/projected/e1cd4bce-415e-4dc5-b31d-076678f97295-kube-api-access-g4dd2\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:36:26.058487 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.058493    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1cd4bce-415e-4dc5-b31d-076678f97295-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:36:26.325902 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.325864    2574 generic.go:356] "Generic (PLEG): container finished" podID="e1cd4bce-415e-4dc5-b31d-076678f97295" containerID="6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362" exitCode=0
Sep 10 16:36:26.326084 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.325903    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-6mr46"
Sep 10 16:36:26.326084 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.325951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-6mr46" event={"ID":"e1cd4bce-415e-4dc5-b31d-076678f97295","Type":"ContainerDied","Data":"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"}
Sep 10 16:36:26.326084 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.326000    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-6mr46" event={"ID":"e1cd4bce-415e-4dc5-b31d-076678f97295","Type":"ContainerDied","Data":"ca27f5c391aa8501cb2b519026c6002949f1c515a77cc9099aab2c7beca4058a"}
Sep 10 16:36:26.326084 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.326028    2574 scope.go:117] "RemoveContainer" containerID="6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"
Sep 10 16:36:26.335180 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.335159    2574 scope.go:117] "RemoveContainer" containerID="6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"
Sep 10 16:36:26.335477 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:36:26.335458    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362\": container with ID starting with 6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362 not found: ID does not exist" containerID="6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"
Sep 10 16:36:26.335547 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.335485    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362"} err="failed to get container status \"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362\": rpc error: code = NotFound desc = could not find container \"6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362\": container with ID starting with 6ecabc08c05a598e9fdc5f90fa24c7ec56b1bef205ff139e2d24d71cf590a362 not found: ID does not exist"
Sep 10 16:36:26.350613 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.350589    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:26.357342 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:26.357304    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-6mr46"]
Sep 10 16:36:27.266644 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.266612    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1cd4bce-415e-4dc5-b31d-076678f97295" path="/var/lib/kubelet/pods/e1cd4bce-415e-4dc5-b31d-076678f97295/volumes"
Sep 10 16:36:27.554131 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.554093    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-ccf7f868f-k6f7f"]
Sep 10 16:36:27.554527 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.554510    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1cd4bce-415e-4dc5-b31d-076678f97295" containerName="manager"
Sep 10 16:36:27.554579 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.554530    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1cd4bce-415e-4dc5-b31d-076678f97295" containerName="manager"
Sep 10 16:36:27.554615 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.554597    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1cd4bce-415e-4dc5-b31d-076678f97295" containerName="manager"
Sep 10 16:36:27.558869 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.558852    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.562254 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.562231    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:36:27.563279 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.563264    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-phs7m\""
Sep 10 16:36:27.563398 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.563279    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:36:27.563398 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.563352    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:36:27.598792 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.598764    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-k6f7f"]
Sep 10 16:36:27.674565 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.674530    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-webhook-cert\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.674741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.674573    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzk9h\" (UniqueName: \"kubernetes.io/projected/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-kube-api-access-vzk9h\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.674741 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.674602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-metrics-certs\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.776023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.775988    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-webhook-cert\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.776023 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.776037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzk9h\" (UniqueName: \"kubernetes.io/projected/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-kube-api-access-vzk9h\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.776235 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.776169    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-metrics-certs\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.778592 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.778567    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-webhook-cert\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.778706 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.778599    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-metrics-certs\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.793144 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.793122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzk9h\" (UniqueName: \"kubernetes.io/projected/91fdd9fe-2ec5-4d35-8eba-31d1746e7009-kube-api-access-vzk9h\") pod \"maas-controller-ccf7f868f-k6f7f\" (UID: \"91fdd9fe-2ec5-4d35-8eba-31d1746e7009\") " pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:27.870114 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:27.870013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:28.146648 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:28.146566    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-ccf7f868f-k6f7f"]
Sep 10 16:36:28.148754 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:36:28.148720    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fdd9fe_2ec5_4d35_8eba_31d1746e7009.slice/crio-3100e055f7ef9b366b447c79eb62b5912ec4ce39ab0824994545ef0a61957af5 WatchSource:0}: Error finding container 3100e055f7ef9b366b447c79eb62b5912ec4ce39ab0824994545ef0a61957af5: Status 404 returned error can't find the container with id 3100e055f7ef9b366b447c79eb62b5912ec4ce39ab0824994545ef0a61957af5
Sep 10 16:36:28.336663 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:28.336624    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-k6f7f" event={"ID":"91fdd9fe-2ec5-4d35-8eba-31d1746e7009","Type":"ContainerStarted","Data":"3100e055f7ef9b366b447c79eb62b5912ec4ce39ab0824994545ef0a61957af5"}
Sep 10 16:36:29.343763 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:29.343719    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-ccf7f868f-k6f7f" event={"ID":"91fdd9fe-2ec5-4d35-8eba-31d1746e7009","Type":"ContainerStarted","Data":"4401e7513820eb3cae1091d6364c7cee64919c9ad862fa0887b6ea2e83c4df2e"}
Sep 10 16:36:29.344200 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:29.343865    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:29.366173 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:29.366117    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-ccf7f868f-k6f7f" podStartSLOduration=2.00884102 podStartE2EDuration="2.366101428s" totalImagesPullingTime="357.260408ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:36:29.364752223 +0000 UTC m=+2084.597700327" watchObservedRunningTime="2026-09-10 16:36:29.366101428 +0000 UTC m=+2084.599049533"
Sep 10 16:36:40.350057 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:40.350022    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-ccf7f868f-k6f7f"
Sep 10 16:36:45.336039 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:45.336005    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:36:45.339681 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:36:45.339645    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:40:34.624898 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.624862    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"]
Sep 10 16:40:34.628046 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.628027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.648941 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.648905    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"]
Sep 10 16:40:34.725004 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.724965    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-maas-api-tls\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.725164 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.725030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2b67e0-24db-4493-b433-b0fdfad59c25-openshift-service-ca\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.725164 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.725109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-metrics-certs\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.725264 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.725164    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7dm4\" (UniqueName: \"kubernetes.io/projected/2e2b67e0-24db-4493-b433-b0fdfad59c25-kube-api-access-l7dm4\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.825642 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.825610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7dm4\" (UniqueName: \"kubernetes.io/projected/2e2b67e0-24db-4493-b433-b0fdfad59c25-kube-api-access-l7dm4\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.825779 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.825654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-maas-api-tls\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.825779 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.825697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2b67e0-24db-4493-b433-b0fdfad59c25-openshift-service-ca\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.825779 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.825733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-metrics-certs\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.826375 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.826323    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2b67e0-24db-4493-b433-b0fdfad59c25-openshift-service-ca\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.828404 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.828379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-metrics-certs\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.828404 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.828391    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2b67e0-24db-4493-b433-b0fdfad59c25-maas-api-tls\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.847399 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.847369    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7dm4\" (UniqueName: \"kubernetes.io/projected/2e2b67e0-24db-4493-b433-b0fdfad59c25-kube-api-access-l7dm4\") pod \"maas-api-75bdcfcf57-6dqb2\" (UID: \"2e2b67e0-24db-4493-b433-b0fdfad59c25\") " pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:34.937793 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:34.937730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:35.237008 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:35.236935    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"]
Sep 10 16:40:35.240000 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:40:35.239974    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2b67e0_24db_4493_b433_b0fdfad59c25.slice/crio-9e286c8eadb49944dacf261445949091fe151bec4b168fabaee38bcdbc3396ca WatchSource:0}: Error finding container 9e286c8eadb49944dacf261445949091fe151bec4b168fabaee38bcdbc3396ca: Status 404 returned error can't find the container with id 9e286c8eadb49944dacf261445949091fe151bec4b168fabaee38bcdbc3396ca
Sep 10 16:40:35.412039 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:35.412002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2" event={"ID":"2e2b67e0-24db-4493-b433-b0fdfad59c25","Type":"ContainerStarted","Data":"9e286c8eadb49944dacf261445949091fe151bec4b168fabaee38bcdbc3396ca"}
Sep 10 16:40:36.417456 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:36.417419    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2" event={"ID":"2e2b67e0-24db-4493-b433-b0fdfad59c25","Type":"ContainerStarted","Data":"85ddc13d95ed7b1e484091b1dffe73d725af71c9d7bc382536f85ed127f888cd"}
Sep 10 16:40:36.417820 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:36.417554    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:36.455709 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:36.455658    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2" podStartSLOduration=1.9853475120000001 podStartE2EDuration="2.455643089s" totalImagesPullingTime="470.295577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:40:36.453888499 +0000 UTC m=+2331.686836601" watchObservedRunningTime="2026-09-10 16:40:36.455643089 +0000 UTC m=+2331.688591191"
Sep 10 16:40:42.426784 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:42.426716    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75bdcfcf57-6dqb2"
Sep 10 16:40:50.197172 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:50.197131    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:40:50.197650 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:50.197347    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" podUID="d5f70685-6550-4288-ab90-90b452b7f9ed" containerName="maas-api" containerID="cri-o://3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa" gracePeriod=30
Sep 10 16:40:51.436103 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.436079    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:40:51.487426 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.487397    2574 generic.go:356] "Generic (PLEG): container finished" podID="d5f70685-6550-4288-ab90-90b452b7f9ed" containerID="3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa" exitCode=0
Sep 10 16:40:51.487587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.487436    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc"
Sep 10 16:40:51.487587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.487486    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" event={"ID":"d5f70685-6550-4288-ab90-90b452b7f9ed","Type":"ContainerDied","Data":"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"}
Sep 10 16:40:51.487587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.487542    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b8859f477-mwplc" event={"ID":"d5f70685-6550-4288-ab90-90b452b7f9ed","Type":"ContainerDied","Data":"426af405615bbeaa365d64e01b8b4ed7caa97b7138043b82c119da8c63516f46"}
Sep 10 16:40:51.487587 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.487571    2574 scope.go:117] "RemoveContainer" containerID="3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"
Sep 10 16:40:51.495763 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.495744    2574 scope.go:117] "RemoveContainer" containerID="3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"
Sep 10 16:40:51.496020 ip-10-0-143-169 kubenswrapper[2574]: E0910 16:40:51.496001    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa\": container with ID starting with 3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa not found: ID does not exist" containerID="3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"
Sep 10 16:40:51.496065 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.496028    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa"} err="failed to get container status \"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa\": rpc error: code = NotFound desc = could not find container \"3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa\": container with ID starting with 3c65be9c46c6797660e3349f66ec6695e8b55b0e6e1403b202645320255d81fa not found: ID does not exist"
Sep 10 16:40:51.559383 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx96n\" (UniqueName: \"kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n\") pod \"d5f70685-6550-4288-ab90-90b452b7f9ed\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") "
Sep 10 16:40:51.559489 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559418    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca\") pod \"d5f70685-6550-4288-ab90-90b452b7f9ed\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") "
Sep 10 16:40:51.559489 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559445    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") pod \"d5f70685-6550-4288-ab90-90b452b7f9ed\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") "
Sep 10 16:40:51.559616 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559594    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") pod \"d5f70685-6550-4288-ab90-90b452b7f9ed\" (UID: \"d5f70685-6550-4288-ab90-90b452b7f9ed\") "
Sep 10 16:40:51.559775 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559750    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d5f70685-6550-4288-ab90-90b452b7f9ed" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:40:51.559885 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.559859    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f70685-6550-4288-ab90-90b452b7f9ed-openshift-service-ca\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:40:51.561502 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.561478    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d5f70685-6550-4288-ab90-90b452b7f9ed" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:40:51.561582 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.561497    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d5f70685-6550-4288-ab90-90b452b7f9ed" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:40:51.561582 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.561499    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n" (OuterVolumeSpecName: "kube-api-access-mx96n") pod "d5f70685-6550-4288-ab90-90b452b7f9ed" (UID: "d5f70685-6550-4288-ab90-90b452b7f9ed"). InnerVolumeSpecName "kube-api-access-mx96n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:40:51.660505 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.660479    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-metrics-certs\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:40:51.660505 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.660500    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx96n\" (UniqueName: \"kubernetes.io/projected/d5f70685-6550-4288-ab90-90b452b7f9ed-kube-api-access-mx96n\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:40:51.660653 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.660511    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d5f70685-6550-4288-ab90-90b452b7f9ed-maas-api-tls\") on node \"ip-10-0-143-169.ec2.internal\" DevicePath \"\""
Sep 10 16:40:51.816783 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.816705    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:40:51.822496 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:51.822467    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-b8859f477-mwplc"]
Sep 10 16:40:53.266218 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:40:53.266186    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f70685-6550-4288-ab90-90b452b7f9ed" path="/var/lib/kubelet/pods/d5f70685-6550-4288-ab90-90b452b7f9ed/volumes"
Sep 10 16:41:08.859303 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:08.859271    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854f6b5cfc-krm9b_57751bf1-9b64-4657-9a71-27aafda7848e/authorino/0.log"
Sep 10 16:41:14.920781 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:14.920736    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-ccf7f868f-k6f7f_91fdd9fe-2ec5-4d35-8eba-31d1746e7009/manager/0.log"
Sep 10 16:41:15.549268 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:15.549240    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67779d4d4c-k4dn4_f5a24fcc-3722-4012-a038-5446012342cc/copy-manifests/0.log"
Sep 10 16:41:15.575137 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:15.575109    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67779d4d4c-k4dn4_f5a24fcc-3722-4012-a038-5446012342cc/manager/0.log"
Sep 10 16:41:15.933978 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:15.933888    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75bdcfcf57-6dqb2_2e2b67e0-24db-4493-b433-b0fdfad59c25/maas-api/0.log"
Sep 10 16:41:17.123292 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:17.123260    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854f6b5cfc-krm9b_57751bf1-9b64-4657-9a71-27aafda7848e/authorino/0.log"
Sep 10 16:41:17.574459 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:17.574427    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-mj82g_5494698d-d5b0-494d-8485-4cf9293657e6/kuadrant-console-plugin/0.log"
Sep 10 16:41:18.666755 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:18.666715    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74797cb87b-xmjxn_fbebe6d2-f166-44ab-8043-27172ba65816/kube-auth-proxy/0.log"
Sep 10 16:41:19.767126 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:19.767085    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-58c8r_321f74f0-b992-46d6-a8a6-4091350db6e8/main/0.log"
Sep 10 16:41:19.995718 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:19.995684    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fv4bzr_794ac9e1-a70f-4c1d-b1d1-3e3ce0a7fa13/main/0.log"
Sep 10 16:41:20.244161 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:20.244131    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-h6tmk_16db3c28-5298-4baa-bfd9-2905cfbde717/main/0.log"
Sep 10 16:41:25.519364 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.519304    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dg8f5/must-gather-2rfgs"]
Sep 10 16:41:25.519808 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.519745    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f70685-6550-4288-ab90-90b452b7f9ed" containerName="maas-api"
Sep 10 16:41:25.519808 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.519759    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f70685-6550-4288-ab90-90b452b7f9ed" containerName="maas-api"
Sep 10 16:41:25.519889 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.519814    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f70685-6550-4288-ab90-90b452b7f9ed" containerName="maas-api"
Sep 10 16:41:25.522847 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.522829    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.526390 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.526364    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dg8f5\"/\"default-dockercfg-rxdwj\""
Sep 10 16:41:25.526549 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.526436    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dg8f5\"/\"kube-root-ca.crt\""
Sep 10 16:41:25.526659 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.526645    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dg8f5\"/\"openshift-service-ca.crt\""
Sep 10 16:41:25.540883 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.540853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg8f5/must-gather-2rfgs"]
Sep 10 16:41:25.646698 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.646662    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-must-gather-output\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.646698 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.646696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kd2p\" (UniqueName: \"kubernetes.io/projected/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-kube-api-access-4kd2p\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.747551 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.747514    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-must-gather-output\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.747551 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.747548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kd2p\" (UniqueName: \"kubernetes.io/projected/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-kube-api-access-4kd2p\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.747934 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.747910    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-must-gather-output\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.761908 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.761877    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kd2p\" (UniqueName: \"kubernetes.io/projected/6b9bc5de-d6fc-4e1a-87c8-101581d920c4-kube-api-access-4kd2p\") pod \"must-gather-2rfgs\" (UID: \"6b9bc5de-d6fc-4e1a-87c8-101581d920c4\") " pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:25.832279 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:25.832249    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg8f5/must-gather-2rfgs"
Sep 10 16:41:26.114056 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:26.114029    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg8f5/must-gather-2rfgs"]
Sep 10 16:41:26.116411 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:41:26.116379    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b9bc5de_d6fc_4e1a_87c8_101581d920c4.slice/crio-4be8815ec41d7f9c944b2ce1d32e4b4c3a24a938a9ea33db01635216f53fbab8 WatchSource:0}: Error finding container 4be8815ec41d7f9c944b2ce1d32e4b4c3a24a938a9ea33db01635216f53fbab8: Status 404 returned error can't find the container with id 4be8815ec41d7f9c944b2ce1d32e4b4c3a24a938a9ea33db01635216f53fbab8
Sep 10 16:41:26.118650 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:26.118630    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:41:26.642080 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:26.642039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg8f5/must-gather-2rfgs" event={"ID":"6b9bc5de-d6fc-4e1a-87c8-101581d920c4","Type":"ContainerStarted","Data":"4be8815ec41d7f9c944b2ce1d32e4b4c3a24a938a9ea33db01635216f53fbab8"}
Sep 10 16:41:27.651981 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:27.651932    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg8f5/must-gather-2rfgs" event={"ID":"6b9bc5de-d6fc-4e1a-87c8-101581d920c4","Type":"ContainerStarted","Data":"cecf8ec985197cb23f9740f1bbfde276ac5a725b82a9283ac1d14847c419f146"}
Sep 10 16:41:27.652526 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:27.651997    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg8f5/must-gather-2rfgs" event={"ID":"6b9bc5de-d6fc-4e1a-87c8-101581d920c4","Type":"ContainerStarted","Data":"897b950d6da7ae9d0ceb8f4f4166dfc9be3a90741a47b4c9d689e7820534f5f7"}
Sep 10 16:41:27.674302 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:27.674244    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dg8f5/must-gather-2rfgs" podStartSLOduration=1.881346454 podStartE2EDuration="2.674222919s" totalImagesPullingTime="792.876465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:41:27.672546028 +0000 UTC m=+2382.905494131" watchObservedRunningTime="2026-09-10 16:41:27.674222919 +0000 UTC m=+2382.907171022"
Sep 10 16:41:29.111749 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:29.111714    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c7krj_e130d30f-5fe3-4bcd-825b-c52b6b9a6e27/global-pull-secret-syncer/0.log"
Sep 10 16:41:29.394144 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:29.394057    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vp6vj_e8ebdd40-30cf-4864-9b3a-1827cbc9d72a/konnectivity-agent/0.log"
Sep 10 16:41:29.508419 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:29.508382    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-169.ec2.internal_4e52fe6a08309506396b2cf1a56b350d/haproxy/0.log"
Sep 10 16:41:34.562930 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:34.562885    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-854f6b5cfc-krm9b_57751bf1-9b64-4657-9a71-27aafda7848e/authorino/0.log"
Sep 10 16:41:34.755438 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:34.755397    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-mj82g_5494698d-d5b0-494d-8485-4cf9293657e6/kuadrant-console-plugin/0.log"
Sep 10 16:41:36.980058 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:36.980018    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bb8cc74d9-ndlnz_842fad69-61de-4835-b3ba-3837faba2c9e/metrics-server/0.log"
Sep 10 16:41:37.026299 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:37.026204    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-x7fn9_29496c12-2be0-411a-a331-ac408d8ebfd5/monitoring-plugin/0.log"
Sep 10 16:41:37.192179 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:37.192144    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j2pgl_75e633de-eb82-4864-9a7e-ecabc4b64bce/node-exporter/0.log"
Sep 10 16:41:37.230593 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:37.230502    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j2pgl_75e633de-eb82-4864-9a7e-ecabc4b64bce/kube-rbac-proxy/0.log"
Sep 10 16:41:37.276837 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:37.276656    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j2pgl_75e633de-eb82-4864-9a7e-ecabc4b64bce/init-textfile/0.log"
Sep 10 16:41:38.722980 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.722947    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"]
Sep 10 16:41:38.729162 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.729126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.748318 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.748274    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"]
Sep 10 16:41:38.781047 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.781003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-proc\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.781406 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.781375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-sys\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.781544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.781416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-lib-modules\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.781544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.781451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkxj\" (UniqueName: \"kubernetes.io/projected/b0e7553c-207d-457a-87db-0f6cfd199226-kube-api-access-grkxj\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.781544 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.781527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-podres\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882534 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-proc\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882579    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-sys\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-proc\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-lib-modules\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkxj\" (UniqueName: \"kubernetes.io/projected/b0e7553c-207d-457a-87db-0f6cfd199226-kube-api-access-grkxj\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882714    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-sys\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-podres\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.882780 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-lib-modules\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.883171 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.882886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b0e7553c-207d-457a-87db-0f6cfd199226-podres\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:38.898821 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:38.898778    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkxj\" (UniqueName: \"kubernetes.io/projected/b0e7553c-207d-457a-87db-0f6cfd199226-kube-api-access-grkxj\") pod \"perf-node-gather-daemonset-m59r6\" (UID: \"b0e7553c-207d-457a-87db-0f6cfd199226\") " pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:39.044574 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.044526    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:39.352176 ip-10-0-143-169 kubenswrapper[2574]: W0910 16:41:39.352138    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb0e7553c_207d_457a_87db_0f6cfd199226.slice/crio-bab442b4774980f67c708030b6d09b7ef528a05e18b785299ff679749e18e94c WatchSource:0}: Error finding container bab442b4774980f67c708030b6d09b7ef528a05e18b785299ff679749e18e94c: Status 404 returned error can't find the container with id bab442b4774980f67c708030b6d09b7ef528a05e18b785299ff679749e18e94c
Sep 10 16:41:39.353693 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.353659    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"]
Sep 10 16:41:39.729750 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.729651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6" event={"ID":"b0e7553c-207d-457a-87db-0f6cfd199226","Type":"ContainerStarted","Data":"bde4c967d3ed1e36a23f72b47adc95fbf7839eb6a2351357da06be8a67fe7648"}
Sep 10 16:41:39.729750 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.729709    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6" event={"ID":"b0e7553c-207d-457a-87db-0f6cfd199226","Type":"ContainerStarted","Data":"bab442b4774980f67c708030b6d09b7ef528a05e18b785299ff679749e18e94c"}
Sep 10 16:41:39.729750 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.729732    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:39.760638 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:39.760570    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6" podStartSLOduration=1.7605501700000001 podStartE2EDuration="1.76055017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:41:39.759210884 +0000 UTC m=+2394.992158987" watchObservedRunningTime="2026-09-10 16:41:39.76055017 +0000 UTC m=+2394.993498271"
Sep 10 16:41:43.085608 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:43.085568    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z4t22_4b3b5327-2865-4fdf-9b5a-be0f85204661/dns/0.log"
Sep 10 16:41:43.115043 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:43.115011    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z4t22_4b3b5327-2865-4fdf-9b5a-be0f85204661/kube-rbac-proxy/0.log"
Sep 10 16:41:43.212700 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:43.212668    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z8cg7_c2ed7b68-a01f-4353-a4ed-e0faeef067ad/dns-node-resolver/0.log"
Sep 10 16:41:43.923190 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:43.923161    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ms7g8_9c4fbf6c-bef9-40d1-ba57-83a6f1ecd27c/node-ca/0.log"
Sep 10 16:41:45.368143 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:45.368112    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:41:45.371221 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:45.371192    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pskgr_a782a69a-5e9e-4eec-9add-50617b03773d/ovn-acl-logging/0.log"
Sep 10 16:41:45.408513 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:45.408484    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74797cb87b-xmjxn_fbebe6d2-f166-44ab-8043-27172ba65816/kube-auth-proxy/0.log"
Sep 10 16:41:45.747527 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:45.747438    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dg8f5/perf-node-gather-daemonset-m59r6"
Sep 10 16:41:46.614839 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:46.614792    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v7njq_52ec76f1-3c0f-4241-b6b5-03eb131577e4/serve-healthcheck-canary/0.log"
Sep 10 16:41:47.389563 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:47.389530    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2tkb_8d3ddef0-fd8c-4346-8b70-7cad9d7362bb/kube-rbac-proxy/0.log"
Sep 10 16:41:47.440314 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:47.440277    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2tkb_8d3ddef0-fd8c-4346-8b70-7cad9d7362bb/exporter/0.log"
Sep 10 16:41:47.495814 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:47.495783    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k2tkb_8d3ddef0-fd8c-4346-8b70-7cad9d7362bb/extractor/0.log"
Sep 10 16:41:49.652014 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:49.651986    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74797cb87b-xmjxn_fbebe6d2-f166-44ab-8043-27172ba65816/kube-auth-proxy/0.log"
Sep 10 16:41:50.741182 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:50.741152    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-ccf7f868f-k6f7f_91fdd9fe-2ec5-4d35-8eba-31d1746e7009/manager/0.log"
Sep 10 16:41:51.165585 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:51.165550    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67779d4d4c-k4dn4_f5a24fcc-3722-4012-a038-5446012342cc/manager/0.log"
Sep 10 16:41:51.185236 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:51.185208    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67779d4d4c-k4dn4_f5a24fcc-3722-4012-a038-5446012342cc/copy-manifests/0.log"
Sep 10 16:41:51.691822 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:51.691790    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74797cb87b-xmjxn_fbebe6d2-f166-44ab-8043-27172ba65816/kube-auth-proxy/0.log"
Sep 10 16:41:52.631652 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:52.631613    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84479dc8dd-vhlpl_a783d7f6-3895-4d0c-aea7-dc1b08af7a8d/manager/0.log"
Sep 10 16:41:52.673472 ip-10-0-143-169 kubenswrapper[2574]: I0910 16:41:52.673442    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tjmx4_58fa5d15-ad01-4864-8abb-d7d8afef4ea4/openshift-lws-operator/0.log"