Sep 10 19:13:39.648463 ip-10-0-134-26 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 19:13:39.648472 ip-10-0-134-26 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 19:13:39.648479 ip-10-0-134-26 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 19:13:39.648690 ip-10-0-134-26 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 19:13:49.898028 ip-10-0-134-26 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 19:13:49.898045 ip-10-0-134-26 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e909d40a0b0841faa78425940e6b4f3d --
Sep 10 19:15:41.072298 ip-10-0-134-26 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 19:15:41.499487 ip-10-0-134-26 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:41.499487 ip-10-0-134-26 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 19:15:41.499487 ip-10-0-134-26 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:41.499487 ip-10-0-134-26 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 19:15:41.499487 ip-10-0-134-26 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 19:15:41.500282 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.499544    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 19:15:41.505600 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505579    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.505600 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505602    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.505600 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505607    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505612    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505616    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505620    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505623    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505628    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505632    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505635    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505639    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505643    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505647    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505651    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505654    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505658    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505662    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505666    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505670    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505674    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505677    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505681    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.505816 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505685    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505689    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505693    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505697    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505701    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505705    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505709    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505713    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505717    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505721    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505726    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505730    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505734    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505738    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505744    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505748    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505753    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505757    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505762    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.506651 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505766    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505770    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505777    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505784    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505789    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505794    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505799    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505807    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505814    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505819    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505823    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505827    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505832    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505837    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505842    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505846    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505850    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505854    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505859    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.507462 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505864    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505867    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505872    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505876    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505880    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505885    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505890    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505894    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505899    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505904    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505908    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505912    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505917    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505921    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505925    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505947    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505954    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505959    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505964    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505968    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.508296 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505973    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505977    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505982    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505987    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505991    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.505996    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506000    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506004    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506008    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506013    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506017    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506020    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506025    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506029    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506033    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506037    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506042    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506046    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.506052    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507187    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.509142 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507198    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507202    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507210    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507216    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507222    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507227    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507232    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507236    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507240    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507244    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507248    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507253    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507257    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507261    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507265    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507269    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507274    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507278    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507282    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507286    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.509916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507291    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507294    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507298    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507303    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507306    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507311    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507315    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507319    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507323    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507327    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507331    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507336    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507342    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507346    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507350    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507355    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507359    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507363    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507367    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507371    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.510466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507375    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507379    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507383    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507386    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507391    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507395    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507399    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507404    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507408    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507412    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507416    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507420    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507424    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507428    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507434    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507438    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507442    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507447    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507451    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507455    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.510984 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507458    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507462    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507466    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507470    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507476    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507480    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507485    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507490    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507497    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507503    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507508    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507513    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507517    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507521    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507525    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507529    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507533    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507538    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507542    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.511489 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507546    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507550    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507554    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507558    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507563    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507569    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507575    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507579    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507584    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507588    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507593    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507597    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507602    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507606    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507610    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507614    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507618    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507624    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.507629    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.511972 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509953    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509966    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509971    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509974    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509977    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509980    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509983    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509986    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509989    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509992    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509994    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.509997    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510000    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510003    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510006    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510008    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510011    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510014    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510017    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510019    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.512532 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510021    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510024    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510027    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510030    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510034    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510039    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510042    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510044    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510048    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510051    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510053    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510056    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510065    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510070    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510073    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510076    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510078    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510081    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510084    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.513058 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510086    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510089    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510092    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510094    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510097    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510099    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510102    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510104    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510107    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510110    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510112    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510115    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510117    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510120    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510123    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510126    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510128    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510131    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510133    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.513617 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510136    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510138    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510142    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510144    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510147    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510149    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510152    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510155    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510158    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510160    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510163    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510166    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510168    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510171    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510173    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510175    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510178    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510180    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510183    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510185    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.514178 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510188    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510190    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510193    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510195    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510198    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510202    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510204    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510207    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510210    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510212    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510215    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510217    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510220    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510222    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510225    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510227    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510230    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510232    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510239    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510244    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.514673 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510247    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510329    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510370    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510378    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510382    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510387    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510390    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510399    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510405    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510408    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510411    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510415    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510418    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510421    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510424    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510427    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510430    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510433    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510437    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510442    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510444    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510447    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510452    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510455    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 19:15:41.515204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510459    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510462    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510465    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510469    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510472    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510475    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510479    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510483    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510485    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510489    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510493    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510496    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510498    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510502    2567 flags.go:64] FLAG: --enable-server="true"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510505    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510509    2567 flags.go:64] FLAG: --event-burst="100"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510513    2567 flags.go:64] FLAG: --event-qps="50"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510516    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510519    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510522    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510526    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510528    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510531    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510534    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510537    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 19:15:41.515778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510540    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510543    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510546    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510549    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510552    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510555    2567 flags.go:64] FLAG: --feature-gates=""
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510559    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510562    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510565    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510568    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510571    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510574    2567 flags.go:64] FLAG: --help="false"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510576    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510580    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510583    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510586    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510589    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510592    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510595    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510598    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510601    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510604    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510607    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510610    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 19:15:41.516458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510613    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510616    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510618    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510621    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510624    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510627    2567 flags.go:64] FLAG: --lock-file=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510629    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510632    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510635    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510640    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510643    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510646    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510649    2567 flags.go:64] FLAG: --logging-format="text"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510653    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510656    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510659    2567 flags.go:64] FLAG: --manifest-url=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510662    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510666    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510669    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510673    2567 flags.go:64] FLAG: --max-pods="110"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510676    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510679    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510682    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510685    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510687    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 19:15:41.517119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510690    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510693    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510700    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510703    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510706    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510709    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510712    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510717    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510720    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510723    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510726    2567 flags.go:64] FLAG: --port="10250"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510729    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510732    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ba6a514323f6a12c"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510735    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510738    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510741    2567 flags.go:64] FLAG: --register-node="true"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510744    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510748    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510751    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510754    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510757    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510761    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510764    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510767    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510770    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 19:15:41.517748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510773    2567 flags.go:64] FLAG: --runonce="false"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510776    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510779    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510782    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510785    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510788    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510791    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510794    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510797    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510800    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510802    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510806    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510809    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510812    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510815    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510817    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510823    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510826    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510829    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510833    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510836    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510838    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510841    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510844    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510847    2567 flags.go:64] FLAG: --v="2"
Sep 10 19:15:41.518386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510851    2567 flags.go:64] FLAG: --version="false"
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510855    2567 flags.go:64] FLAG: --vmodule=""
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510859    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.510862    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510988    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510992    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510996    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.510999    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511001    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511004    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511007    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511011    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511014    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511017    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511019    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511022    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511025    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511028    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511030    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511033    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.519062 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511036    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511039    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511041    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511044    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511046    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511049    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511051    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511054    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511057    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511060    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511062    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511065    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511069    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511072    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511078    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511081    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511084    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511087    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511089    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.519552 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511092    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511095    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511098    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511100    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511104    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511107    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511110    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511113    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511115    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511118    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511120    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511123    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511126    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511128    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511131    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511133    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511136    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511138    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511141    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511143    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.520029 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511146    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511148    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511151    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511154    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511156    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511159    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511161    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511165    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511168    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511171    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511174    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511176    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511179    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511181    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511184    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511187    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511190    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511193    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511195    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511198    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.520512 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511201    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511203    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511206    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511209    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511211    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511214    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511216    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511219    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511222    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511224    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511227    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511229    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511232    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511234    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511237    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511240    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511244    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511248    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511252    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.521094 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511257    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511259    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511262    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511269    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.511271    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.512015    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.519702    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.519719    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.520617    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.520626    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520683    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520688    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520691    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520695    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520698    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.521570 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520702    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520705    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520708    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520711    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520714    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520717    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520719    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520723    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520727    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520731    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520734    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520736    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520739    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520742    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520745    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520748    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520750    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520753    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520755    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.521962 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520758    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520761    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520763    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520766    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520768    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520773    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520776    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520779    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520782    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520784    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520787    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520790    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520793    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520796    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520798    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520801    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520804    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520807    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520809    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520812    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.522461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520815    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520817    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520820    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520823    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520826    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520829    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520831    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520834    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520837    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520840    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520843    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520845    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520848    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520851    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520853    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520856    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520858    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520861    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520864    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520867    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.522965 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520870    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520874    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520877    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520880    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520883    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520887    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520890    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520892    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520895    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520897    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520900    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520903    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520906    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520910    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520912    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520915    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520918    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520920    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520923    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.523449 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520926    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520929    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520945    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520949    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520951    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520954    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520957    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520960    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520963    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520965    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520968    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520971    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520974    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520976    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520979    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.523903 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.520982    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.520988    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521098    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521103    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521106    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521109    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521112    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521115    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521118    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521121    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521124    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521126    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521129    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521131    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521134    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 19:15:41.524294 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521137    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521139    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521142    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521145    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521148    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521151    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521153    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521156    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521158    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521161    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521164    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521166    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521169    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521172    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521174    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521178    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521180    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521183    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521186    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521188    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 19:15:41.524662 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521191    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521193    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521196    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521199    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521201    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521204    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521206    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521210    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521213    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521216    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521219    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521222    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521225    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521228    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521230    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521233    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521236    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521239    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521241    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 19:15:41.525172 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521244    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521246    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521249    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521251    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521254    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521256    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521260    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521263    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521266    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521270    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521273    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521275    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521277    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521280    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521283    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521285    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521288    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521290    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521293    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 19:15:41.525631 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521295    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521298    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521300    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521303    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521305    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521308    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521311    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521313    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521316    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521318    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521322    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521324    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521327    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521331    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521334    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521337    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521339    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521342    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521345    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521347    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 19:15:41.526110 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521350    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521353    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521355    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521358    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521361    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521363    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521366    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:41.521369    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.521374    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.521490    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.523865    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.524969    2567 server.go:1019] "Starting client certificate rotation"
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.525067    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:41.526574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.525097    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 19:15:41.527872 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.527858    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:41.528972 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.528954    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 19:15:41.543664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.543643    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 10 19:15:41.546987 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.546973    2567 log.go:25] "Validated CRI v1 image API"
Sep 10 19:15:41.547855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.547840    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 19:15:41.551262 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.551243    2567 fs.go:135] Filesystem UUIDs: map[75418c5e-782f-4695-b0e8-43fe5aecc91b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ceb16429-78b9-422e-87ec-ddcc458d6fce:/dev/nvme0n1p4]
Sep 10 19:15:41.551312 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.551261    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 19:15:41.557068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.556950    2567 manager.go:216] Machine: {Timestamp:2026-09-10 19:15:41.55643149 +0000 UTC m=+0.377487615 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3120388 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ef27393dbd23d70b0529db9b71c96 SystemUUID:ec2ef273-93db-d23d-70b0-529db9b71c96 BootID:e909d40a-0b08-41fa-a784-25940e6b4f3d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:78:27:86:c9:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:78:27:86:c9:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:ae:ce:80:42:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 19:15:41.557068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.557057    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 19:15:41.557182 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.557166    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 19:15:41.560119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.560090    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 19:15:41.560255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.560122    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-26.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 19:15:41.560991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.560980    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 19:15:41.561023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.560993    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 19:15:41.561023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.561002    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:41.561995 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.561984    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 19:15:41.563805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.563794    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 19:15:41.568465 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568453    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:41.568581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568572    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 19:15:41.568614 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568608    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 19:15:41.568642 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568622    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 19:15:41.568668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568645    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 19:15:41.568668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568659    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 10 19:15:41.568717 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568670    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 19:15:41.569006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.568995    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:41.569062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.569012    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 19:15:41.571395 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.571368    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 19:15:41.572768 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.572753    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:41.574466 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574444    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 19:15:41.574532 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574474    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 19:15:41.574633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574623    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 19:15:41.574666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574639    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 19:15:41.574666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574645    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 19:15:41.574666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574650    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 19:15:41.574666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574660    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 19:15:41.574779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.574669    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 19:15:41.575284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575274    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 19:15:41.575326 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575285    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 19:15:41.575326 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575293    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 19:15:41.575326 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575298    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 19:15:41.575326 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575312    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 19:15:41.575326 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.575320    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 19:15:41.576134 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.576124    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 19:15:41.576134 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.576134    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 19:15:41.579446 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.579424    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-26.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 19:15:41.579446 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.579427    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 19:15:41.579553 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.579427    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 19:15:41.580219 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.580208    2567 server.go:1268] "Started kubelet"
Sep 10 19:15:41.580328 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.580290    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 19:15:41.580379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.580315    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 19:15:41.580379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.580350    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 19:15:41.580965 ip-10-0-134-26 systemd[1]: Started Kubernetes Kubelet.
Sep 10 19:15:41.581903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.581889    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 19:15:41.582591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.582579    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 19:15:41.587313 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.587289    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 19:15:41.588759 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.587517    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-26.ec2.internal.18d40ba1cfff4226  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-26.ec2.internal,UID:ip-10-0-134-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-26.ec2.internal,},FirstTimestamp:2026-09-10 19:15:41.580182054 +0000 UTC m=+0.401238178,LastTimestamp:2026-09-10 19:15:41.580182054 +0000 UTC m=+0.401238178,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-26.ec2.internal,}"
Sep 10 19:15:41.589898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.589883    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 19:15:41.589898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.589891    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:41.590048 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.590031    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.590664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590648    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 19:15:41.590756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590674    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 19:15:41.590756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590699    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 19:15:41.590857 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590808    2567 factory.go:55] Registering systemd factory
Sep 10 19:15:41.590857 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590828    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 10 19:15:41.590970 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590894    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 19:15:41.590970 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.590901    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 19:15:41.591111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591094    2567 factory.go:153] Registering CRI-O factory
Sep 10 19:15:41.591111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591113    2567 factory.go:223] Registration of the crio container factory successfully
Sep 10 19:15:41.591250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591163    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 19:15:41.591250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591181    2567 factory.go:103] Registering Raw factory
Sep 10 19:15:41.591250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591198    2567 manager.go:1196] Started watching for new ooms in manager
Sep 10 19:15:41.591704 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.591693    2567 manager.go:349] Starting recovery of all containers
Sep 10 19:15:41.594148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.594119    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 19:15:41.594407 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.594379    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-26.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 19:15:41.605152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.605135    2567 manager.go:354] Recovery completed
Sep 10 19:15:41.606826 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.606805    2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 10 19:15:41.610137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.610125    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.611805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.611781    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.611873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.611817    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.611873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.611828    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.612419 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.612405    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 19:15:41.612419 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.612417    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 19:15:41.612540 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.612428    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.612540 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.612435    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 19:15:41.613168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.613153    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.613219 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.613182    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.613219 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.613194    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.614335 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.614257    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-26.ec2.internal.18d40ba1d1e1c674  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-26.ec2.internal,UID:ip-10-0-134-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-26.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-26.ec2.internal,},FirstTimestamp:2026-09-10 19:15:41.611804276 +0000 UTC m=+0.432860399,LastTimestamp:2026-09-10 19:15:41.611804276 +0000 UTC m=+0.432860399,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-26.ec2.internal,}"
Sep 10 19:15:41.614584 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.614572    2567 policy_none.go:49] "None policy: Start"
Sep 10 19:15:41.614642 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.614590    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 19:15:41.614642 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.614602    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 19:15:41.616690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.616677    2567 policy_none.go:47] "Start"
Sep 10 19:15:41.618480 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.618460    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7vpvx"
Sep 10 19:15:41.622893 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.622803    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-26.ec2.internal.18d40ba1d1e20d1e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-26.ec2.internal,UID:ip-10-0-134-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-26.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-26.ec2.internal,},FirstTimestamp:2026-09-10 19:15:41.611822366 +0000 UTC m=+0.432878490,LastTimestamp:2026-09-10 19:15:41.611822366 +0000 UTC m=+0.432878490,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-26.ec2.internal,}"
Sep 10 19:15:41.624604 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.624548    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-26.ec2.internal.18d40ba1d1e23115  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-26.ec2.internal,UID:ip-10-0-134-26.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-26.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-26.ec2.internal,},FirstTimestamp:2026-09-10 19:15:41.611831573 +0000 UTC m=+0.432887698,LastTimestamp:2026-09-10 19:15:41.611831573 +0000 UTC m=+0.432887698,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-26.ec2.internal,}"
Sep 10 19:15:41.627488 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.627472    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7vpvx"
Sep 10 19:15:41.653593 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.653575    2567 manager.go:337] "Starting Device Plugin manager"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.653620    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.653634    2567 server.go:87] "Starting device plugin registration server"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.653917    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.653946    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.654308    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.654385    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.654393    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.654968    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 19:15:41.671859 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.655014    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.694034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.693987    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 19:15:41.695208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.695191    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 19:15:41.695276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.695240    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 19:15:41.695276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.695263    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 19:15:41.695369 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.695299    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 19:15:41.697382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.697363    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:41.754136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.754026    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.755055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.755035    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.755164 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.755065    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.755164 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.755075    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.755164 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.755099    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.764059 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.764044    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.764102 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.764067    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-26.ec2.internal\": node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.787585 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.787560    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.795963 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.795928    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"]
Sep 10 19:15:41.796122 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.796028    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.797376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.797361    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.797467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.797384    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.797467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.797393    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.799692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.799680    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.799809 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.799796    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.799846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.799827    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.800411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800392    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.800411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800406    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.800580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800424    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.800580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800430    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.800580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800437    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.800580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.800443    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.802668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.802655    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.802714 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.802681    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:41.803362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.803343    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:41.803439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.803371    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:41.803439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.803382    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:41.821510 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.821493    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-26.ec2.internal\" not found" node="ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.826016 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.826002    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-26.ec2.internal\" not found" node="ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.887980 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.887954    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.892702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.892683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c918b9dd0fb780c2b3cbfafa24767b4e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-26.ec2.internal\" (UID: \"c918b9dd0fb780c2b3cbfafa24767b4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.892772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.892710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.892772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.892728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.988243 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:41.988200    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:41.993626 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.993681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c918b9dd0fb780c2b3cbfafa24767b4e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-26.ec2.internal\" (UID: \"c918b9dd0fb780c2b3cbfafa24767b4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.993681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.993681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993676    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.993780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993722    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c918b9dd0fb780c2b3cbfafa24767b4e-config\") pod \"kube-apiserver-proxy-ip-10-0-134-26.ec2.internal\" (UID: \"c918b9dd0fb780c2b3cbfafa24767b4e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:41.993780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:41.993760    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb3f891ab567f26ad1d723f321b14a0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal\" (UID: \"cb3f891ab567f26ad1d723f321b14a0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:42.089146 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.089069    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.124558 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.124530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:42.128486 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.128459    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:42.189411 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.189237    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.295327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.295269    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.395835 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.395745    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.419333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.419301    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:42.419808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.419792    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:42.496603 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.496566    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.525926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.525900    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 19:15:42.526386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.526074    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:42.526386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.526114    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:42.526386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.526129    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 19:15:42.597211 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.597184    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.631424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.631384    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 19:10:42 +0000 UTC" deadline="2028-04-16 07:31:47.27625145 +0000 UTC"
Sep 10 19:15:42.631424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.631417    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14004h16m4.644838548s"
Sep 10 19:15:42.654313 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.654245    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 19:15:42.655360 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.655340    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 19:15:42.655458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.655372    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 19:15:42.655458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.655387    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeHasSufficientPID"
Sep 10 19:15:42.698254 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.698229    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.799195 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.799158    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.900363 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:42.900322    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:42.919991 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:42.919943    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc918b9dd0fb780c2b3cbfafa24767b4e.slice/crio-9160b32a18dc2a3e7203bd47b5e43c2b2f02a464b89d54ee40398db7fff018ac WatchSource:0}: Error finding container 9160b32a18dc2a3e7203bd47b5e43c2b2f02a464b89d54ee40398db7fff018ac: Status 404 returned error can't find the container with id 9160b32a18dc2a3e7203bd47b5e43c2b2f02a464b89d54ee40398db7fff018ac
Sep 10 19:15:42.920260 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:42.920231    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3f891ab567f26ad1d723f321b14a0e.slice/crio-e946e5e4e1c71b08a29c32f53ed42e58d221bcba14f49bb016c911ecac666843 WatchSource:0}: Error finding container e946e5e4e1c71b08a29c32f53ed42e58d221bcba14f49bb016c911ecac666843: Status 404 returned error can't find the container with id e946e5e4e1c71b08a29c32f53ed42e58d221bcba14f49bb016c911ecac666843
Sep 10 19:15:42.927284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:42.927261    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:15:43.000761 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.000711    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:43.101528 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.101478    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-26.ec2.internal\" not found"
Sep 10 19:15:43.113514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.113487    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:43.193193 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.193120    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:43.210211 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.210186    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:43.211196 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.211182    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal"
Sep 10 19:15:43.220484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.220465    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 19:15:43.572277 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.572196    2567 apiserver.go:52] "Watching apiserver"
Sep 10 19:15:43.581808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.581787    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 19:15:43.582835 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.582815    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-wprrs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4","openshift-dns/node-resolver-ck8rf","openshift-network-diagnostics/network-check-target-4qdbk","openshift-network-operator/iptables-alerter-wgsgb","openshift-ovn-kubernetes/ovnkube-node-r7vb4","kube-system/global-pull-secret-syncer-n7lpb","kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jr6zr","openshift-image-registry/node-ca-lbmvr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal","openshift-multus/multus-additional-cni-plugins-wxv8f","openshift-multus/multus-fzhnf","openshift-multus/network-metrics-daemon-6qj78"]
Sep 10 19:15:43.586818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.586803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.586886 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.586868    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:43.589034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.589019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.590883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.590855    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 19:15:43.591320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591091    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.591320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591129    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 19:15:43.591320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591161    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.591320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591202    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gjcr8\""
Sep 10 19:15:43.591590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591422    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 19:15:43.591590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.591538    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.593597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.593576    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.593707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.593671    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4zrjm\""
Sep 10 19:15:43.593875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.593857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:43.593940 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.593919    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:43.594021 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.593861    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.596157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.596139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.598545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.598509    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 19:15:43.598648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.598579    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pcj9f\""
Sep 10 19:15:43.598921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.598757    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.601306 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.600671    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.601306 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.600758    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.602885 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.602857    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 19:15:43.603103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603077    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tj896\""
Sep 10 19:15:43.603407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-multus-certs\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603423    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df856401-4913-4059-9bce-5bdf0655c1bd-tmp-dir\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.603524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603390    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 19:15:43.603524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld9q9\" (UniqueName: \"kubernetes.io/projected/df856401-4913-4059-9bce-5bdf0655c1bd-kube-api-access-ld9q9\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.603689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-kubelet-config\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.603689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-system-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-k8s-cni-cncf-io\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603646    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-conf-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-multus-daemon-config\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-etc-kubernetes\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-cnibin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-cni-binary-copy\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-multus\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-kubelet\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.603903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-hostroot\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54w6\" (UniqueName: \"kubernetes.io/projected/24170250-ffbb-4c0d-b409-e47efe4993be-kube-api-access-h54w6\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df856401-4913-4059-9bce-5bdf0655c1bd-hosts-file\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.603974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-netns\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-bin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604017    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-dbus\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.604190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-socket-dir-parent\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.604651 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.605191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604996    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.605191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.605020    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 19:15:43.605191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.604979    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 19:15:43.605191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.605046    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 19:15:43.605191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.605052    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.606402 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.606384    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.606830 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.606809    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.606955 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.606854    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.606955 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.606919    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-djhw8\""
Sep 10 19:15:43.609055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.609033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4mkp5\""
Sep 10 19:15:43.609160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.609145    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 19:15:43.609160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.609153    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 19:15:43.609387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.609374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.611658 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.611619    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v6xb6\""
Sep 10 19:15:43.611750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.611657    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 19:15:43.611750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.611664    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.611902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.611885    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.612238 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.612223    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.614119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.614094    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 19:15:43.614304 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.614289    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 19:15:43.614406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.614351    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 19:15:43.614455 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.614424    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r45qd\""
Sep 10 19:15:43.614509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.614461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.614553 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.614527    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:43.617109 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.617094    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.619266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.619249    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 19:15:43.619342 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.619252    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 19:15:43.619517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.619502    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wngt5\""
Sep 10 19:15:43.637290 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.637268    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wm6tl"
Sep 10 19:15:43.648914 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.648895    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wm6tl"
Sep 10 19:15:43.691655 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.691630    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 19:15:43.699430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.699144    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:43.701714 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.701652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal" event={"ID":"cb3f891ab567f26ad1d723f321b14a0e","Type":"ContainerStarted","Data":"e946e5e4e1c71b08a29c32f53ed42e58d221bcba14f49bb016c911ecac666843"}
Sep 10 19:15:43.702812 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.702789    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal" event={"ID":"c918b9dd0fb780c2b3cbfafa24767b4e","Type":"ContainerStarted","Data":"9160b32a18dc2a3e7203bd47b5e43c2b2f02a464b89d54ee40398db7fff018ac"}
Sep 10 19:15:43.704410 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704392    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.704523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-kubelet-config\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.704523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-system-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.704523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704454    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-hostroot\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.704523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704470    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/264a5601-4ce3-4a62-9baf-0f9c175a4244-serviceca\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95578d05-bf3f-4168-8ce1-3e38c9601bc3-konnectivity-ca\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-hostroot\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-kubelet-config\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-netns\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704603    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-systemd-units\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-system-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-ovn\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-bin\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0154061b-3847-436f-87bc-7fe4c8e4c399-ovn-node-metrics-cert\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.704715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df856401-4913-4059-9bce-5bdf0655c1bd-tmp-dir\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-systemd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-etc-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704785    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znb2n\" (UniqueName: \"kubernetes.io/projected/0154061b-3847-436f-87bc-7fe4c8e4c399-kube-api-access-znb2n\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95578d05-bf3f-4168-8ce1-3e38c9601bc3-agent-certs\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-cnibin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-cni-binary-copy\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-conf\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-run\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704983    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-cnibin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.704983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-netns\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705009    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-bin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-netns\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-bin\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705094    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/264a5601-4ce3-4a62-9baf-0f9c175a4244-host\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlfp4\" (UniqueName: \"kubernetes.io/projected/10ee0884-afe5-449b-8b47-406b7c26b7cc-kube-api-access-zlfp4\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.705212    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-kubernetes\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-tmp\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.705279    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:44.205256926 +0000 UTC m=+3.026313062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705338    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ee0884-afe5-449b-8b47-406b7c26b7cc-host-slash\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-netd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-config\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qgm8\" (UniqueName: \"kubernetes.io/projected/264a5601-4ce3-4a62-9baf-0f9c175a4244-kube-api-access-9qgm8\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-var-lib-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.705609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705470    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-cni-binary-copy\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54w6\" (UniqueName: \"kubernetes.io/projected/24170250-ffbb-4c0d-b409-e47efe4993be-kube-api-access-h54w6\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-conf-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-multus-daemon-config\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-etc-kubernetes\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-etc-kubernetes\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-conf-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705598    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df856401-4913-4059-9bce-5bdf0655c1bd-tmp-dir\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysconfig\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-env-overrides\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-var-lib-kubelet\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705729    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-etc-selinux\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-sys-fs\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-dbus\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-socket-dir-parent\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-etc-tuned\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.706257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twhrl\" (UniqueName: \"kubernetes.io/projected/7a79710f-8fa6-4e87-a30b-e661cb50d797-kube-api-access-twhrl\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-multus-certs\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705923    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5trhl\" (UniqueName: \"kubernetes.io/projected/5817f7ec-d60b-4f3c-9e24-8aa71b716536-kube-api-access-5trhl\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10ee0884-afe5-449b-8b47-406b7c26b7cc-iptables-alerter-script\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705964    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-socket-dir-parent\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705993    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/24170250-ffbb-4c0d-b409-e47efe4993be-multus-daemon-config\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.705981    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-kubelet\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706018    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed5274b-92ef-48e0-9466-a04023ae4f93-dbus\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-modprobe-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-multus-certs\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-systemd\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706093    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-sys\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df856401-4913-4059-9bce-5bdf0655c1bd-hosts-file\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld9q9\" (UniqueName: \"kubernetes.io/projected/df856401-4913-4059-9bce-5bdf0655c1bd-kube-api-access-ld9q9\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-k8s-cni-cncf-io\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-lib-modules\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-host\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.707160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706189    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df856401-4913-4059-9bce-5bdf0655c1bd-hosts-file\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-socket-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-run-k8s-cni-cncf-io\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-multus\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706301    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-multus-cni-dir\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-cni-multus\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706329    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-kubelet\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24170250-ffbb-4c0d-b409-e47efe4993be-host-var-lib-kubelet\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhhbn\" (UniqueName: \"kubernetes.io/projected/f535af16-3f8b-43f9-9c39-59b1a86b9080-kube-api-access-bhhbn\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cnibin\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g2pq\" (UniqueName: \"kubernetes.io/projected/6e361013-79fc-4110-9a70-4eb4550021c7-kube-api-access-9g2pq\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-registration-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706448    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-system-cni-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-node-log\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.708160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.708960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.708960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-slash\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.708960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-log-socket\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.708960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-script-lib\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.708960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.706595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-device-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.718428 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.718407    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 19:15:43.722351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.722333    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld9q9\" (UniqueName: \"kubernetes.io/projected/df856401-4913-4059-9bce-5bdf0655c1bd-kube-api-access-ld9q9\") pod \"node-resolver-ck8rf\" (UID: \"df856401-4913-4059-9bce-5bdf0655c1bd\") " pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.722458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.722359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54w6\" (UniqueName: \"kubernetes.io/projected/24170250-ffbb-4c0d-b409-e47efe4993be-kube-api-access-h54w6\") pod \"multus-fzhnf\" (UID: \"24170250-ffbb-4c0d-b409-e47efe4993be\") " pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.807556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.807556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-systemd-units\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-ovn\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-bin\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807646    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0154061b-3847-436f-87bc-7fe4c8e4c399-ovn-node-metrics-cert\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-systemd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-etc-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znb2n\" (UniqueName: \"kubernetes.io/projected/0154061b-3847-436f-87bc-7fe4c8e4c399-kube-api-access-znb2n\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807762    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95578d05-bf3f-4168-8ce1-3e38c9601bc3-agent-certs\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807790    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-conf\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.807855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-run\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/264a5601-4ce3-4a62-9baf-0f9c175a4244-host\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlfp4\" (UniqueName: \"kubernetes.io/projected/10ee0884-afe5-449b-8b47-406b7c26b7cc-kube-api-access-zlfp4\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.807967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-kubernetes\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-tmp\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808053    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ee0884-afe5-449b-8b47-406b7c26b7cc-host-slash\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-netd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-config\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qgm8\" (UniqueName: \"kubernetes.io/projected/264a5601-4ce3-4a62-9baf-0f9c175a4244-kube-api-access-9qgm8\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808138    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-var-lib-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysconfig\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-env-overrides\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-var-lib-kubelet\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.808363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-etc-selinux\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-sys-fs\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-etc-tuned\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808273    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twhrl\" (UniqueName: \"kubernetes.io/projected/7a79710f-8fa6-4e87-a30b-e661cb50d797-kube-api-access-twhrl\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5trhl\" (UniqueName: \"kubernetes.io/projected/5817f7ec-d60b-4f3c-9e24-8aa71b716536-kube-api-access-5trhl\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10ee0884-afe5-449b-8b47-406b7c26b7cc-iptables-alerter-script\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-kubelet\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-kubernetes\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808390    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-modprobe-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808407    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-systemd\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-systemd-units\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-sys\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-ovn\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808457    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-lib-modules\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-bin\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-host\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-socket-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808525    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-host\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhhbn\" (UniqueName: \"kubernetes.io/projected/f535af16-3f8b-43f9-9c39-59b1a86b9080-kube-api-access-bhhbn\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cnibin\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g2pq\" (UniqueName: \"kubernetes.io/projected/6e361013-79fc-4110-9a70-4eb4550021c7-kube-api-access-9g2pq\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-registration-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-system-cni-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808658    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-node-log\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-slash\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-log-socket\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808781    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-script-lib\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-device-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-device-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.809980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808938    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-socket-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.808966    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/264a5601-4ce3-4a62-9baf-0f9c175a4244-serviceca\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809010    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95578d05-bf3f-4168-8ce1-3e38c9601bc3-konnectivity-ca\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809040    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-netns\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809132    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-netns\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cnibin\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-registration-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809509    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-system-cni-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-systemd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-etc-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.809997    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10ee0884-afe5-449b-8b47-406b7c26b7cc-host-slash\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.810045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-node-log\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.810074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-cni-netd\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.810128    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.810227    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:44.310207604 +0000 UTC m=+3.131263722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:43.810856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.810674    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-config\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.810925    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5817f7ec-d60b-4f3c-9e24-8aa71b716536-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.810989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-slash\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-var-lib-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-log-socket\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-run-openvswitch\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811179    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysconfig\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811269    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-tmp\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/264a5601-4ce3-4a62-9baf-0f9c175a4244-host\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-run-ovn-kubernetes\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-ovnkube-script-lib\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-sysctl-conf\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.811727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811731    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-run\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-var-lib-kubelet\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-etc-selinux\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.811972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a79710f-8fa6-4e87-a30b-e661cb50d797-sys-fs\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.812705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/264a5601-4ce3-4a62-9baf-0f9c175a4244-serviceca\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.812812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/95578d05-bf3f-4168-8ce1-3e38c9601bc3-konnectivity-ca\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.812826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-modprobe-d\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.812898    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0154061b-3847-436f-87bc-7fe4c8e4c399-host-kubelet\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.813323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.813069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5817f7ec-d60b-4f3c-9e24-8aa71b716536-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.813778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.813366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/10ee0884-afe5-449b-8b47-406b7c26b7cc-iptables-alerter-script\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.813778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.813452    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-sys\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.813778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.813506    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-etc-systemd\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.813778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.813567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e361013-79fc-4110-9a70-4eb4550021c7-lib-modules\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.814861 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.814630    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0154061b-3847-436f-87bc-7fe4c8e4c399-env-overrides\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.816098 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.816012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0154061b-3847-436f-87bc-7fe4c8e4c399-ovn-node-metrics-cert\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.817948 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.817900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/95578d05-bf3f-4168-8ce1-3e38c9601bc3-agent-certs\") pod \"konnectivity-agent-wprrs\" (UID: \"95578d05-bf3f-4168-8ce1-3e38c9601bc3\") " pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.818754 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.818731    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e361013-79fc-4110-9a70-4eb4550021c7-etc-tuned\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.820056 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.819720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhhbn\" (UniqueName: \"kubernetes.io/projected/f535af16-3f8b-43f9-9c39-59b1a86b9080-kube-api-access-bhhbn\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:43.820237 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.820208    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:43.820333 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.820258    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:43.820333 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.820274    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:43.820455 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:43.820340    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:15:44.320321682 +0000 UTC m=+3.141377809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:43.820530 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.820517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g2pq\" (UniqueName: \"kubernetes.io/projected/6e361013-79fc-4110-9a70-4eb4550021c7-kube-api-access-9g2pq\") pod \"tuned-jr6zr\" (UID: \"6e361013-79fc-4110-9a70-4eb4550021c7\") " pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.821813 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.821786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qgm8\" (UniqueName: \"kubernetes.io/projected/264a5601-4ce3-4a62-9baf-0f9c175a4244-kube-api-access-9qgm8\") pod \"node-ca-lbmvr\" (UID: \"264a5601-4ce3-4a62-9baf-0f9c175a4244\") " pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.822571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.822415    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znb2n\" (UniqueName: \"kubernetes.io/projected/0154061b-3847-436f-87bc-7fe4c8e4c399-kube-api-access-znb2n\") pod \"ovnkube-node-r7vb4\" (UID: \"0154061b-3847-436f-87bc-7fe4c8e4c399\") " pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.822668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.822619    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlfp4\" (UniqueName: \"kubernetes.io/projected/10ee0884-afe5-449b-8b47-406b7c26b7cc-kube-api-access-zlfp4\") pod \"iptables-alerter-wgsgb\" (UID: \"10ee0884-afe5-449b-8b47-406b7c26b7cc\") " pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.823136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.823114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twhrl\" (UniqueName: \"kubernetes.io/projected/7a79710f-8fa6-4e87-a30b-e661cb50d797-kube-api-access-twhrl\") pod \"aws-ebs-csi-driver-node-hp5t4\" (UID: \"7a79710f-8fa6-4e87-a30b-e661cb50d797\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.823389 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.823366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5trhl\" (UniqueName: \"kubernetes.io/projected/5817f7ec-d60b-4f3c-9e24-8aa71b716536-kube-api-access-5trhl\") pod \"multus-additional-cni-plugins-wxv8f\" (UID: \"5817f7ec-d60b-4f3c-9e24-8aa71b716536\") " pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.901235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.901193    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fzhnf"
Sep 10 19:15:43.907945 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.907923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ck8rf"
Sep 10 19:15:43.913746 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.913711    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24170250_ffbb_4c0d_b409_e47efe4993be.slice/crio-cc5d848788692fb28c17f20a91cf182de68f78105980170d457cd9c7960f7f04 WatchSource:0}: Error finding container cc5d848788692fb28c17f20a91cf182de68f78105980170d457cd9c7960f7f04: Status 404 returned error can't find the container with id cc5d848788692fb28c17f20a91cf182de68f78105980170d457cd9c7960f7f04
Sep 10 19:15:43.916602 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.916579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgsgb"
Sep 10 19:15:43.920027 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.919990    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf856401_4913_4059_9bce_5bdf0655c1bd.slice/crio-1c1566e2fe8708af23420e657839ab325d15bbcb104322cf9679080e39ac35bc WatchSource:0}: Error finding container 1c1566e2fe8708af23420e657839ab325d15bbcb104322cf9679080e39ac35bc: Status 404 returned error can't find the container with id 1c1566e2fe8708af23420e657839ab325d15bbcb104322cf9679080e39ac35bc
Sep 10 19:15:43.922129 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.922113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:15:43.926851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.926834    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jr6zr"
Sep 10 19:15:43.927144 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.927104    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ee0884_afe5_449b_8b47_406b7c26b7cc.slice/crio-3fca726469e71402df4a667b42e8513bd99e6e1db71f62c7e16bc417c0a58346 WatchSource:0}: Error finding container 3fca726469e71402df4a667b42e8513bd99e6e1db71f62c7e16bc417c0a58346: Status 404 returned error can't find the container with id 3fca726469e71402df4a667b42e8513bd99e6e1db71f62c7e16bc417c0a58346
Sep 10 19:15:43.933120 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.933098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:15:43.935632 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.935607    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0154061b_3847_436f_87bc_7fe4c8e4c399.slice/crio-17c7509eee52c87e11339defd4d5dc31802dc8f823ee1a45b0da47dd446480e8 WatchSource:0}: Error finding container 17c7509eee52c87e11339defd4d5dc31802dc8f823ee1a45b0da47dd446480e8: Status 404 returned error can't find the container with id 17c7509eee52c87e11339defd4d5dc31802dc8f823ee1a45b0da47dd446480e8
Sep 10 19:15:43.938532 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.938500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4"
Sep 10 19:15:43.942077 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.942053    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e361013_79fc_4110_9a70_4eb4550021c7.slice/crio-0bd2a5589fc663885d73a7c496c90c8e91f526f91aaee90f3c8bc4b5fa2dcc9d WatchSource:0}: Error finding container 0bd2a5589fc663885d73a7c496c90c8e91f526f91aaee90f3c8bc4b5fa2dcc9d: Status 404 returned error can't find the container with id 0bd2a5589fc663885d73a7c496c90c8e91f526f91aaee90f3c8bc4b5fa2dcc9d
Sep 10 19:15:43.944205 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.943992    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lbmvr"
Sep 10 19:15:43.947403 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.947376    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95578d05_bf3f_4168_8ce1_3e38c9601bc3.slice/crio-9edd8eb82216d032c5bc159e3551c7f1863b7dc84d9f39b9de1430ebf848daf1 WatchSource:0}: Error finding container 9edd8eb82216d032c5bc159e3551c7f1863b7dc84d9f39b9de1430ebf848daf1: Status 404 returned error can't find the container with id 9edd8eb82216d032c5bc159e3551c7f1863b7dc84d9f39b9de1430ebf848daf1
Sep 10 19:15:43.948771 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:43.948752    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wxv8f"
Sep 10 19:15:43.952256 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.951856    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a79710f_8fa6_4e87_a30b_e661cb50d797.slice/crio-cff3a290bc2a7537a1715cb460bd143574854a8386f2941def9f7037f338d28a WatchSource:0}: Error finding container cff3a290bc2a7537a1715cb460bd143574854a8386f2941def9f7037f338d28a: Status 404 returned error can't find the container with id cff3a290bc2a7537a1715cb460bd143574854a8386f2941def9f7037f338d28a
Sep 10 19:15:43.966287 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.966254    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod264a5601_4ce3_4a62_9baf_0f9c175a4244.slice/crio-b11d378d9a5e80562194003f4874bd7f15e4f1435f1027db62f86c3c4a46e975 WatchSource:0}: Error finding container b11d378d9a5e80562194003f4874bd7f15e4f1435f1027db62f86c3c4a46e975: Status 404 returned error can't find the container with id b11d378d9a5e80562194003f4874bd7f15e4f1435f1027db62f86c3c4a46e975
Sep 10 19:15:43.967771 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:15:43.967740    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5817f7ec_d60b_4f3c_9e24_8aa71b716536.slice/crio-29277c9c3daad406573a4850cf86ad99bead456cfd3c340cb46ae4f2e78778ab WatchSource:0}: Error finding container 29277c9c3daad406573a4850cf86ad99bead456cfd3c340cb46ae4f2e78778ab: Status 404 returned error can't find the container with id 29277c9c3daad406573a4850cf86ad99bead456cfd3c340cb46ae4f2e78778ab
Sep 10 19:15:44.212589 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.212491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:44.212773 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.212617    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:44.212773 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.212675    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:45.212662117 +0000 UTC m=+4.033718229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:44.303385 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.303354    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:44.313848 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.313606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:44.313848 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.313755    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:44.313848 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.313820    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:45.313799179 +0000 UTC m=+4.134855314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:44.415096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.415054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:44.415261 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.415242    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:44.415340 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.415265    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:44.415340 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.415280    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:44.415459 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:44.415345    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:15:45.415326269 +0000 UTC m=+4.236382405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:44.652202 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.652031    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:44 +0000 UTC" deadline="2028-06-16 03:35:48.88112298 +0000 UTC"
Sep 10 19:15:44.652202 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.652091    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15464h20m4.229037234s"
Sep 10 19:15:44.707384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.707349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"17c7509eee52c87e11339defd4d5dc31802dc8f823ee1a45b0da47dd446480e8"}
Sep 10 19:15:44.708927 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.708878    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgsgb" event={"ID":"10ee0884-afe5-449b-8b47-406b7c26b7cc","Type":"ContainerStarted","Data":"3fca726469e71402df4a667b42e8513bd99e6e1db71f62c7e16bc417c0a58346"}
Sep 10 19:15:44.710275 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.710242    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fzhnf" event={"ID":"24170250-ffbb-4c0d-b409-e47efe4993be","Type":"ContainerStarted","Data":"cc5d848788692fb28c17f20a91cf182de68f78105980170d457cd9c7960f7f04"}
Sep 10 19:15:44.711756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.711729    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerStarted","Data":"29277c9c3daad406573a4850cf86ad99bead456cfd3c340cb46ae4f2e78778ab"}
Sep 10 19:15:44.714228 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.714202    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jr6zr" event={"ID":"6e361013-79fc-4110-9a70-4eb4550021c7","Type":"ContainerStarted","Data":"0bd2a5589fc663885d73a7c496c90c8e91f526f91aaee90f3c8bc4b5fa2dcc9d"}
Sep 10 19:15:44.715571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.715544    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ck8rf" event={"ID":"df856401-4913-4059-9bce-5bdf0655c1bd","Type":"ContainerStarted","Data":"1c1566e2fe8708af23420e657839ab325d15bbcb104322cf9679080e39ac35bc"}
Sep 10 19:15:44.717036 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.717009    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lbmvr" event={"ID":"264a5601-4ce3-4a62-9baf-0f9c175a4244","Type":"ContainerStarted","Data":"b11d378d9a5e80562194003f4874bd7f15e4f1435f1027db62f86c3c4a46e975"}
Sep 10 19:15:44.718893 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.718865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4" event={"ID":"7a79710f-8fa6-4e87-a30b-e661cb50d797","Type":"ContainerStarted","Data":"cff3a290bc2a7537a1715cb460bd143574854a8386f2941def9f7037f338d28a"}
Sep 10 19:15:44.720311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:44.720284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wprrs" event={"ID":"95578d05-bf3f-4168-8ce1-3e38c9601bc3","Type":"ContainerStarted","Data":"9edd8eb82216d032c5bc159e3551c7f1863b7dc84d9f39b9de1430ebf848daf1"}
Sep 10 19:15:45.067076 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.066952    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 19:15:45.221491 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.221450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:45.221665 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.221625    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:45.221740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.221693    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.221674014 +0000 UTC m=+6.042730129 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:45.323054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.322338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:45.323054 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.322567    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:45.323054 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.322633    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.322614428 +0000 UTC m=+6.143670547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:45.423788 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.423099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:45.423788 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.423339    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:45.423788 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.423363    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:45.423788 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.423376    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:45.423788 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.423437    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:15:47.42341939 +0000 UTC m=+6.244475508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:45.652376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.652250    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 19:10:44 +0000 UTC" deadline="2028-05-22 10:33:32.301251884 +0000 UTC"
Sep 10 19:15:45.652376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.652291    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14871h17m46.648965809s"
Sep 10 19:15:45.698981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.698548    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:45.698981 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.698876    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:45.699556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.699330    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:45.699556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:45.699357    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:45.699556 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.699440    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:45.699556 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:45.699521    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:47.252586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.252543    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:47.253109 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.252715    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.253109 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.252783    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:51.252763709 +0000 UTC m=+10.073819828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:47.353735 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.353687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:47.353912 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.353852    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.354026 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.353917    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:51.353898519 +0000 UTC m=+10.174954641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:47.454747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.454709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:47.454971 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.454925    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:47.455049 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.454947    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:47.455049 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.455006    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.455170 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.455085    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:15:51.455054143 +0000 UTC m=+10.276110280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.698460    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.698610    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.698990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.699105    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:47.699156    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:47.699371 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:47.699224    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:49.696989 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:49.696880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:49.697478 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:49.697043    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:49.697478 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:49.697138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:49.697478 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:49.697243    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:49.697478 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:49.697296    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:49.697478 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:49.697382    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:51.288969 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.288345    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:51.288969 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.288545    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:51.288969 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.288609    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:59.288588033 +0000 UTC m=+18.109644172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:51.389567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.389533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:51.389738 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.389719    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:51.389807 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.389794    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:15:59.38977142 +0000 UTC m=+18.210827536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:51.491002 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.490940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:51.491199 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.491098    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:51.491199 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.491129    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:51.491199 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.491143    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:51.491320 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.491213    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:15:59.491191595 +0000 UTC m=+18.312247712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:51.696489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.696454    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:51.696658 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.696569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:51.696741 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.696696    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:51.696741 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.696568    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:51.696862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:51.696758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:51.696862 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:51.696834    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:53.695952 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:53.695906    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:53.696392 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:53.696070    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:53.696392 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:53.696300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:53.696509 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:53.696422    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:53.696509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:53.696452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:53.696599 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:53.696547    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:55.696048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:55.696013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:55.696528 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:55.696017    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:55.696528 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:55.696165    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:55.696528 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:55.696027    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:55.696528 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:55.696214    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:55.696528 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:55.696339    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:57.696416 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:57.696379    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:57.696852 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:57.696379    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:57.696852 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:57.696510    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:15:57.696852 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:57.696610    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:57.696852 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:57.696380    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:57.696852 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:57.696675    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:59.352409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.352369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:59.352885 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.352516    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:59.352885 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.352590    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:15.352573001 +0000 UTC m=+34.173629133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:15:59.453546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.453503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:59.453733 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.453687    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:59.453799 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.453772    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:15.453748575 +0000 UTC m=+34.274804701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:15:59.554578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.554535    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:59.554767 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.554699    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:15:59.554767 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.554722    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:15:59.554767 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.554736    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:59.554914 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.554794    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:16:15.554778979 +0000 UTC m=+34.375835090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:15:59.696548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.696463    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:15:59.696548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.696482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:15:59.696548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:15:59.696463    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:15:59.696815 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.696606    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:15:59.696815 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.696664    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:15:59.696815 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:15:59.696743    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:01.696841 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:01.696804    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:01.697316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:01.696917    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:01.697316 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:01.696986    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:01.697316 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:01.696989    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:01.697316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:01.697079    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:01.697316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:01.697190    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:03.696619 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.696279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:03.697215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.696279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:03.697215 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:03.696706    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:03.697215 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:03.696811    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:03.697215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.696323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:03.697215 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:03.696924    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:03.762621 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.762587    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal" event={"ID":"c918b9dd0fb780c2b3cbfafa24767b4e","Type":"ContainerStarted","Data":"a470660cbd4fe64821849ea2da2bd07a482492758278b4e97c3fc655e2ee5a4b"}
Sep 10 19:16:03.764162 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.764133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lbmvr" event={"ID":"264a5601-4ce3-4a62-9baf-0f9c175a4244","Type":"ContainerStarted","Data":"4d254dc881ca5022bbf82672ad68dd979b214aa3d01b52506797b8702ab61ae9"}
Sep 10 19:16:03.765707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.765685    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4" event={"ID":"7a79710f-8fa6-4e87-a30b-e661cb50d797","Type":"ContainerStarted","Data":"682c426a60d419c067e3168ee6a8a5cde3fac587c645b60f85d723616b965102"}
Sep 10 19:16:03.767189 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.767170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wprrs" event={"ID":"95578d05-bf3f-4168-8ce1-3e38c9601bc3","Type":"ContainerStarted","Data":"c24f8866d1529a482f0eb060fcfe0306731e52138f5acfae11a04b2974cead65"}
Sep 10 19:16:03.770054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770038    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:16:03.770418 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770400    2567 generic.go:356] "Generic (PLEG): container finished" podID="0154061b-3847-436f-87bc-7fe4c8e4c399" containerID="d0f0497d8464ddabbe7f6226a16419e32a2afb203f6dfcbdbefdc4c12d09ac1f" exitCode=1
Sep 10 19:16:03.770513 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770452    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"896878e9c76fb3805290d2a7d0a7c9b829121ddb641e2d8abc8cfc2ef90f2f0d"}
Sep 10 19:16:03.770513 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"224f274b245194cd681052f7c46ae0da294046926e871498c72ead39af4bf825"}
Sep 10 19:16:03.770596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770544    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"9249f52bf13c0075eb0db5f841df8564b1d3bd62b008cab804d823ebdd570046"}
Sep 10 19:16:03.770596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770579    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"ede80955c4837e2026c5831d9a738528d34f5e85b0942773af6e2f3f028186c4"}
Sep 10 19:16:03.770666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770608    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerDied","Data":"d0f0497d8464ddabbe7f6226a16419e32a2afb203f6dfcbdbefdc4c12d09ac1f"}
Sep 10 19:16:03.770666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.770640    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"5e6c985b4c3ea8e74addda33bb60bfad93fbd58cb2516e51b9e9ed88f90fd46d"}
Sep 10 19:16:03.771902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.771881    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fzhnf" event={"ID":"24170250-ffbb-4c0d-b409-e47efe4993be","Type":"ContainerStarted","Data":"a4ebdc21e7f9742ae86d0743c10b06c6498ac914d894ad8ed891fb3ccd817eed"}
Sep 10 19:16:03.773265 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.773243    2567 generic.go:356] "Generic (PLEG): container finished" podID="cb3f891ab567f26ad1d723f321b14a0e" containerID="d4bac64b3bcf19fad008311d5fa2727addfcc5e2d0eaf7edb52b0db79177d5b1" exitCode=0
Sep 10 19:16:03.773361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.773329    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal" event={"ID":"cb3f891ab567f26ad1d723f321b14a0e","Type":"ContainerDied","Data":"d4bac64b3bcf19fad008311d5fa2727addfcc5e2d0eaf7edb52b0db79177d5b1"}
Sep 10 19:16:03.774928 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.774909    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="20dd20474881ac5059e06de3012727c6e899f7c5fc990a2a08ce3e1e25be701a" exitCode=0
Sep 10 19:16:03.774928 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.774954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"20dd20474881ac5059e06de3012727c6e899f7c5fc990a2a08ce3e1e25be701a"}
Sep 10 19:16:03.776552 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.776531    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jr6zr" event={"ID":"6e361013-79fc-4110-9a70-4eb4550021c7","Type":"ContainerStarted","Data":"e5bd48ff4108a4109b40275e04059a8743361ee29c84e85105a9ee599c082deb"}
Sep 10 19:16:03.778184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.778166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ck8rf" event={"ID":"df856401-4913-4059-9bce-5bdf0655c1bd","Type":"ContainerStarted","Data":"656d766be752385c85314d94526849ae5d379a3d6b1e05f4112bfd0f702b5da2"}
Sep 10 19:16:03.792191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.792135    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-26.ec2.internal" podStartSLOduration=20.792120538 podStartE2EDuration="20.792120538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.777589423 +0000 UTC m=+22.598645556" watchObservedRunningTime="2026-09-10 19:16:03.792120538 +0000 UTC m=+22.613176671"
Sep 10 19:16:03.792602 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.792569    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wprrs" podStartSLOduration=2.815560466 podStartE2EDuration="21.792561268s" totalImagesPullingTime="18.977000802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.792023568 +0000 UTC m=+22.613079701" watchObservedRunningTime="2026-09-10 19:16:03.792561268 +0000 UTC m=+22.613617402"
Sep 10 19:16:03.807049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.806921    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lbmvr" podStartSLOduration=2.848006886 podStartE2EDuration="21.806903209s" totalImagesPullingTime="18.958896323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.806513627 +0000 UTC m=+22.627569759" watchObservedRunningTime="2026-09-10 19:16:03.806903209 +0000 UTC m=+22.627959344"
Sep 10 19:16:03.862807 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.862763    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fzhnf" podStartSLOduration=2.651353328 podStartE2EDuration="21.86274789s" totalImagesPullingTime="19.211394562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.862291802 +0000 UTC m=+22.683347935" watchObservedRunningTime="2026-09-10 19:16:03.86274789 +0000 UTC m=+22.683804023"
Sep 10 19:16:03.905373 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.905316    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jr6zr" podStartSLOduration=2.923122793 podStartE2EDuration="21.90530017s" totalImagesPullingTime="18.982177377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.881717752 +0000 UTC m=+22.702773888" watchObservedRunningTime="2026-09-10 19:16:03.90530017 +0000 UTC m=+22.726356368"
Sep 10 19:16:03.905715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:03.905695    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ck8rf" podStartSLOduration=6.998063034 podStartE2EDuration="21.905689574s" totalImagesPullingTime="14.90762654s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:03.904591898 +0000 UTC m=+22.725648032" watchObservedRunningTime="2026-09-10 19:16:03.905689574 +0000 UTC m=+22.726745706"
Sep 10 19:16:04.782581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:04.782380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgsgb" event={"ID":"10ee0884-afe5-449b-8b47-406b7c26b7cc","Type":"ContainerStarted","Data":"bf67cb9a8e20eabba5bd514df0a7d7641c045bd55556dfdf9d7cddb919f3848b"}
Sep 10 19:16:04.784694 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:04.784663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal" event={"ID":"cb3f891ab567f26ad1d723f321b14a0e","Type":"ContainerStarted","Data":"63e1bdf04ef882a0b9a2336e4001af1573eedea3ed8223e7cc655c885c4f19d9"}
Sep 10 19:16:04.801448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:04.801407    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wgsgb" podStartSLOduration=3.854172379 podStartE2EDuration="22.801394687s" totalImagesPullingTime="18.947222308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:04.801078654 +0000 UTC m=+23.622134787" watchObservedRunningTime="2026-09-10 19:16:04.801394687 +0000 UTC m=+23.622450819"
Sep 10 19:16:04.815897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:04.815860    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-26.ec2.internal" podStartSLOduration=21.815846165 podStartE2EDuration="21.815846165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:04.815346467 +0000 UTC m=+23.636402601" watchObservedRunningTime="2026-09-10 19:16:04.815846165 +0000 UTC m=+23.636902297"
Sep 10 19:16:04.838498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:04.838478    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 19:16:05.130184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.130108    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:16:05.130781 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.130758    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:16:05.666663 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.666573    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T19:16:04.838495536Z","UUID":"dcec859d-927a-4717-9e5d-32484678ce64","Handler":null,"Name":"","Endpoint":""}
Sep 10 19:16:05.668570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.668543    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 19:16:05.668705 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.668585    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 19:16:05.696452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.696421    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:05.696628 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:05.696557    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:05.696901 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.696884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:05.697029 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:05.697010    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:05.697093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.697046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:05.697174 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:05.697152    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:05.792738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.792309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4" event={"ID":"7a79710f-8fa6-4e87-a30b-e661cb50d797","Type":"ContainerStarted","Data":"5bcfb1d87684269a7f5a9c4952d55b0d3472d4cc8b02d77664441ade376bf3a9"}
Sep 10 19:16:05.792738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.792421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4" event={"ID":"7a79710f-8fa6-4e87-a30b-e661cb50d797","Type":"ContainerStarted","Data":"e04bca4b179f49c951a2f456477b7e0fb7829e832ae90c43776c23adc45e3e10"}
Sep 10 19:16:05.797127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.797107    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:16:05.797794 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.797772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"7b73e6ddd3081e9cf126da9ba439dcb9447720284a7252bab853117beb027212"}
Sep 10 19:16:05.798517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.798386    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:16:05.798840 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.798819    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wprrs"
Sep 10 19:16:05.812035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:05.811982    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hp5t4" podStartSLOduration=6.833532054 podStartE2EDuration="23.811913405s" totalImagesPullingTime="16.978381351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:05.811556991 +0000 UTC m=+24.632613128" watchObservedRunningTime="2026-09-10 19:16:05.811913405 +0000 UTC m=+24.632969539"
Sep 10 19:16:07.696392 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:07.696348    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:07.696866 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:07.696365    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:07.696866 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:07.696478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:07.696866 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:07.696471    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:07.696866 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:07.696591    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:07.696866 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:07.696664    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:08.807575 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.807423    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:16:08.807962 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.807921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"a42763b0a9ceb7e016440a5f68313e74bc891ab765f88c9e16ee522cbd15f627"}
Sep 10 19:16:08.808336 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.808308    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:08.808463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.808367    2567 scope.go:117] "RemoveContainer" containerID="d0f0497d8464ddabbe7f6226a16419e32a2afb203f6dfcbdbefdc4c12d09ac1f"
Sep 10 19:16:08.808463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.808393    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:08.808463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.808441    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:08.810123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.810098    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="9887c8b1d172f67db49ef6c188d8ea1ac0c7c117346c7ba704226ceaf34c7a8d" exitCode=0
Sep 10 19:16:08.810235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.810138    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"9887c8b1d172f67db49ef6c188d8ea1ac0c7c117346c7ba704226ceaf34c7a8d"}
Sep 10 19:16:08.825679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.825655    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:08.826071 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:08.826056    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:09.696172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.696138    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:09.696459 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:09.696297    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:09.696459 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.696308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:09.696583 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:09.696472    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:09.696792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.696766    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:09.696907 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:09.696887    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:09.815855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.815826    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:16:09.816403 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.816165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" event={"ID":"0154061b-3847-436f-87bc-7fe4c8e4c399","Type":"ContainerStarted","Data":"af6dcda3f6755871a96c9929c9f5982ea9d9a343c6ead1e9c3c4c0e18ad12ae1"}
Sep 10 19:16:09.844225 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:09.844169    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4" podStartSLOduration=8.84022221 podStartE2EDuration="27.844152672s" totalImagesPullingTime="19.003930462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:09.843086531 +0000 UTC m=+28.664142664" watchObservedRunningTime="2026-09-10 19:16:09.844152672 +0000 UTC m=+28.665208805"
Sep 10 19:16:10.422821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.422743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qdbk"]
Sep 10 19:16:10.422991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.422900    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:10.423067 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:10.423034    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:10.423481 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.423460    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7lpb"]
Sep 10 19:16:10.423587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.423536    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:10.423648 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:10.423627    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:10.425274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.425253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6qj78"]
Sep 10 19:16:10.425362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.425334    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:10.425455 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:10.425430    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:10.821055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.820973    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="23aa2e240738878c00fc2e0c2ca828672bc2fea9e5a751d53a85aed80d24f79a" exitCode=0
Sep 10 19:16:10.821618 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:10.821057    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"23aa2e240738878c00fc2e0c2ca828672bc2fea9e5a751d53a85aed80d24f79a"}
Sep 10 19:16:11.826156 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:11.826116    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="f143d8cc7d95769c8d7c2a51bff689facee41148d67835c8d06e4402f31943d7" exitCode=0
Sep 10 19:16:11.826607 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:11.826172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"f143d8cc7d95769c8d7c2a51bff689facee41148d67835c8d06e4402f31943d7"}
Sep 10 19:16:12.695855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:12.695821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:12.696072 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:12.695831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:12.696072 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:12.695969    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:12.696072 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:12.695966    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:12.696072 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:12.696047    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:12.696275 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:12.696096    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:14.695831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:14.695791    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:14.695831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:14.695816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:14.696344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:14.695898    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:14.696344 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:14.695910    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qdbk" podUID="203eda7c-4e7a-42cc-b225-ef360158a40b"
Sep 10 19:16:14.696344 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:14.696008    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6qj78" podUID="f535af16-3f8b-43f9-9c39-59b1a86b9080"
Sep 10 19:16:14.696344 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:14.696071    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7lpb" podUID="9ed5274b-92ef-48e0-9466-a04023ae4f93"
Sep 10 19:16:15.370851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:15.370814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:15.371179 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.371007    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:15.371179 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.371084    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret podName:9ed5274b-92ef-48e0-9466-a04023ae4f93 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:47.371063438 +0000 UTC m=+66.192119564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret") pod "global-pull-secret-syncer-n7lpb" (UID: "9ed5274b-92ef-48e0-9466-a04023ae4f93") : object "kube-system"/"original-pull-secret" not registered
Sep 10 19:16:15.471923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:15.471889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:15.472118 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.472027    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:15.472118 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.472079    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs podName:f535af16-3f8b-43f9-9c39-59b1a86b9080 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:47.472066406 +0000 UTC m=+66.293122516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs") pod "network-metrics-daemon-6qj78" (UID: "f535af16-3f8b-43f9-9c39-59b1a86b9080") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 19:16:15.572780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:15.572743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:15.572956 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.572911    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 19:16:15.572956 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.572954    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 19:16:15.573030 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.572967    2567 projected.go:196] Error preparing data for projected volume kube-api-access-9wz7p for pod openshift-network-diagnostics/network-check-target-4qdbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:15.573030 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:15.573024    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p podName:203eda7c-4e7a-42cc-b225-ef360158a40b nodeName:}" failed. No retries permitted until 2026-09-10 19:16:47.573008311 +0000 UTC m=+66.394064432 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9wz7p" (UniqueName: "kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p") pod "network-check-target-4qdbk" (UID: "203eda7c-4e7a-42cc-b225-ef360158a40b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 19:16:16.008992 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.008968    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-26.ec2.internal" event="NodeReady"
Sep 10 19:16:16.009340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.009092    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 19:16:16.049468 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.049437    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"]
Sep 10 19:16:16.073664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.073606    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"]
Sep 10 19:16:16.073824 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.073735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"
Sep 10 19:16:16.076555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.076276    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.076555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.076353    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.076555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.076521    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-czw6s\""
Sep 10 19:16:16.085775 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.085726    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:16:16.086578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.086051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.089835 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.089751    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 19:16:16.089835 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.089819    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pbp9b\""
Sep 10 19:16:16.091177 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.091157    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.091271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.091194    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.091271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.091200    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 19:16:16.104047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.104021    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"]
Sep 10 19:16:16.104176 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.104065    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"]
Sep 10 19:16:16.105095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.104639    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.106714 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.106695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 19:16:16.107030 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.107008    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qxq2w\""
Sep 10 19:16:16.107160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.107008    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 19:16:16.107305 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.107289    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 19:16:16.116294 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.116262    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 19:16:16.117998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.117981    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8rgkr"]
Sep 10 19:16:16.118628 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.118608    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.120725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.120707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 19:16:16.120871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.120842    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.121092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.121076    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.121700 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.121680    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qr7g6\""
Sep 10 19:16:16.124171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.124152    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-psnrk"]
Sep 10 19:16:16.124280 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.124265    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.126722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.126578    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.126722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.126610    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 19:16:16.126722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.126620    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8zmc5\""
Sep 10 19:16:16.126722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.126665    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.127101 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.126788    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 19:16:16.132519 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132500    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"]
Sep 10 19:16:16.132600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132527    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 19:16:16.132600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132535    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:16:16.132600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"]
Sep 10 19:16:16.132751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132601    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.132906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132606    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8rgkr"]
Sep 10 19:16:16.132965 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.132956    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-cff896496-rh6zc"]
Sep 10 19:16:16.134701 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.134679    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 19:16:16.134701 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.134697    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkm4c\""
Sep 10 19:16:16.134842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.134803    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 19:16:16.142664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.142645    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"]
Sep 10 19:16:16.142748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.142739    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.145053 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145035    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 19:16:16.145152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145108    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 19:16:16.145152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145115    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rklhx\""
Sep 10 19:16:16.145260 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145162    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 19:16:16.145646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145624    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 19:16:16.145646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145630    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.145789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.145670    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.150731 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.150709    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-skmtm"]
Sep 10 19:16:16.150825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.150782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.152957 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.152918    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 19:16:16.153046 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.152958    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rrdch\""
Sep 10 19:16:16.153046 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.152995    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.153255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.153236    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 19:16:16.153335 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.153241    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.159582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.159562    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"]
Sep 10 19:16:16.159705 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.159689    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.161772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.161746    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 19:16:16.161893 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.161873    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.162191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.162174    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rjc7l\""
Sep 10 19:16:16.162191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.162187    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.167817 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.167802    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mr9td"]
Sep 10 19:16:16.167953 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.167922    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.170315 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.170299    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.170417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.170333    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 19:16:16.170417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.170406    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 19:16:16.170650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.170636    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.174658 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.174642    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-slcrt\""
Sep 10 19:16:16.177160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.177139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9qhd\" (UniqueName: \"kubernetes.io/projected/ae7517b3-e699-4658-8e2a-919d98953038-kube-api-access-m9qhd\") pod \"volume-data-source-validator-695c895bd6-rpwzs\" (UID: \"ae7517b3-e699-4658-8e2a-919d98953038\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"
Sep 10 19:16:16.177264 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.177168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjpp\" (UniqueName: \"kubernetes.io/projected/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-kube-api-access-fwjpp\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.177264 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.177234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.177361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.177281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.178949 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.178916    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-6r9qv"]
Sep 10 19:16:16.179025 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.179008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.181557 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.181537    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 19:16:16.181648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.181580    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zv8kr\""
Sep 10 19:16:16.181815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.181798    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 19:16:16.187332 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.187302    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-8lvlx"]
Sep 10 19:16:16.187625 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.187607    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.189955 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.189921    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 19:16:16.190047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.190026    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zk299\""
Sep 10 19:16:16.190114 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.190059    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 19:16:16.190214 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.190195    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.190294 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.190227    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.194153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194133    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-psnrk"]
Sep 10 19:16:16.194262 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx"
Sep 10 19:16:16.194412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"]
Sep 10 19:16:16.194480 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-skmtm"]
Sep 10 19:16:16.194480 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194469    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"]
Sep 10 19:16:16.194590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-8lvlx"]
Sep 10 19:16:16.194590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194513    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-6r9qv"]
Sep 10 19:16:16.194590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194532    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mr9td"]
Sep 10 19:16:16.194590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.194550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cff896496-rh6zc"]
Sep 10 19:16:16.195568 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.195550    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 19:16:16.196599 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.196578    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 19:16:16.196693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.196642    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 19:16:16.196786 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.196764    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bh7qh\""
Sep 10 19:16:16.277667 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277727    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-648fx\" (UniqueName: \"kubernetes.io/projected/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-kube-api-access-648fx\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af80a1bb-0779-48f4-9f78-9802c2015d70-config\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9qhd\" (UniqueName: \"kubernetes.io/projected/ae7517b3-e699-4658-8e2a-919d98953038-kube-api-access-m9qhd\") pod \"volume-data-source-validator-695c895bd6-rpwzs\" (UID: \"ae7517b3-e699-4658-8e2a-919d98953038\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-trusted-ca\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.277851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a29dd7c6-d869-4497-9a3a-23a7289a6c58-serving-cert\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pnxq\" (UniqueName: \"kubernetes.io/projected/cb45bf43-4568-4efe-8593-2c61a1950857-kube-api-access-2pnxq\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-default-certificate\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.277993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af80a1bb-0779-48f4-9f78-9802c2015d70-serving-cert\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-cert\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zj5g\" (UniqueName: \"kubernetes.io/projected/af80a1bb-0779-48f4-9f78-9802c2015d70-kube-api-access-4zj5g\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgjt7\" (UniqueName: \"kubernetes.io/projected/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-kube-api-access-qgjt7\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.278184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-config\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-stats-auth\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e6dbc24-2f34-48d9-9233-26f2ab31da70-nginx-conf\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-snapshots\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ddcg\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278497    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjpp\" (UniqueName: \"kubernetes.io/projected/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-kube-api-access-fwjpp\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.278656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2wg7\" (UniqueName: \"kubernetes.io/projected/4c6da106-a66c-443a-8ee2-606a548948af-kube-api-access-h2wg7\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278842    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9s2\" (UniqueName: \"kubernetes.io/projected/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-kube-api-access-gv9s2\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-tmp\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278918    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pllbt\" (UniqueName: \"kubernetes.io/projected/a29dd7c6-d869-4497-9a3a-23a7289a6c58-kube-api-access-pllbt\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.278971    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-serving-cert\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-config-volume\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf5vk\" (UniqueName: \"kubernetes.io/projected/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-kube-api-access-bf5vk\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-tmp-dir\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.279157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.279250 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.279164    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:16.280019 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.279236    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.779216834 +0000 UTC m=+35.600272976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:16.293158 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.293132    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9qhd\" (UniqueName: \"kubernetes.io/projected/ae7517b3-e699-4658-8e2a-919d98953038-kube-api-access-m9qhd\") pod \"volume-data-source-validator-695c895bd6-rpwzs\" (UID: \"ae7517b3-e699-4658-8e2a-919d98953038\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"
Sep 10 19:16:16.293525 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.293503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjpp\" (UniqueName: \"kubernetes.io/projected/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-kube-api-access-fwjpp\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.379584 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-config-volume\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.379748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379597    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgzw\" (UniqueName: \"kubernetes.io/projected/5d16ad53-c0ec-418c-8cdb-45db02f2e073-kube-api-access-lsgzw\") pod \"network-check-source-685456655f-8lvlx\" (UID: \"5d16ad53-c0ec-418c-8cdb-45db02f2e073\") " pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx"
Sep 10 19:16:16.379748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf5vk\" (UniqueName: \"kubernetes.io/projected/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-kube-api-access-bf5vk\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.379748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-tmp-dir\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.379748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-648fx\" (UniqueName: \"kubernetes.io/projected/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-kube-api-access-648fx\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379842    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af80a1bb-0779-48f4-9f78-9802c2015d70-config\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379871    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-trusted-ca\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.379956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.379953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a29dd7c6-d869-4497-9a3a-23a7289a6c58-serving-cert\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pnxq\" (UniqueName: \"kubernetes.io/projected/cb45bf43-4568-4efe-8593-2c61a1950857-kube-api-access-2pnxq\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-default-certificate\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-tmp-dir\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af80a1bb-0779-48f4-9f78-9802c2015d70-serving-cert\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.380153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-cert\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zj5g\" (UniqueName: \"kubernetes.io/projected/af80a1bb-0779-48f4-9f78-9802c2015d70-kube-api-access-4zj5g\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380211    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgjt7\" (UniqueName: \"kubernetes.io/projected/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-kube-api-access-qgjt7\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-config-volume\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380268    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.880252113 +0000 UTC m=+35.701308230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-config\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380318    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:16.380327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380328    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-stats-auth\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380381    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.88037046 +0000 UTC m=+35.701426570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e6dbc24-2f34-48d9-9233-26f2ab31da70-nginx-conf\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-snapshots\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380431    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380469    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.880457327 +0000 UTC m=+35.701513453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ddcg\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380500    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.380525    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.880517045 +0000 UTC m=+35.701573156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.380588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380609    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380662    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2wg7\" (UniqueName: \"kubernetes.io/projected/4c6da106-a66c-443a-8ee2-606a548948af-kube-api-access-h2wg7\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9s2\" (UniqueName: \"kubernetes.io/projected/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-kube-api-access-gv9s2\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-tmp\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380773    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pllbt\" (UniqueName: \"kubernetes.io/projected/a29dd7c6-d869-4497-9a3a-23a7289a6c58-kube-api-access-pllbt\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-serving-cert\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.380957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381072    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381116    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af80a1bb-0779-48f4-9f78-9802c2015d70-config\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381203    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6e6dbc24-2f34-48d9-9233-26f2ab31da70-nginx-conf\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381549    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.381660    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.381722    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.881705677 +0000 UTC m=+35.702761788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:16.381765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.381756    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-config\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.382087 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.382019    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-trusted-ca\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.382310    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.382481    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-snapshots\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.382805    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:16.882788053 +0000 UTC m=+35.703844165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.382991    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a29dd7c6-d869-4497-9a3a-23a7289a6c58-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.383025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.383616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.383276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a29dd7c6-d869-4497-9a3a-23a7289a6c58-tmp\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.384023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.383998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-cert\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.384468 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.384433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-stats-auth\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.385789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.385744    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.385789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.385775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af80a1bb-0779-48f4-9f78-9802c2015d70-serving-cert\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.385963 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.385775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-default-certificate\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.385963 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.385851    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.386380 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.386353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"
Sep 10 19:16:16.386501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.386483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-serving-cert\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.386571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.386537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a29dd7c6-d869-4497-9a3a-23a7289a6c58-serving-cert\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.387231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.387206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.399662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.397172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.399662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.398065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pnxq\" (UniqueName: \"kubernetes.io/projected/cb45bf43-4568-4efe-8593-2c61a1950857-kube-api-access-2pnxq\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.399662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.398094    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgjt7\" (UniqueName: \"kubernetes.io/projected/e5bf6c7c-9f53-4bf9-ab7f-21e695929503-kube-api-access-qgjt7\") pod \"ingress-canary-skmtm\" (UID: \"e5bf6c7c-9f53-4bf9-ab7f-21e695929503\") " pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.399662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.398622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-648fx\" (UniqueName: \"kubernetes.io/projected/9c8d45b7-a264-43de-a9e8-3d8a4fb57669-kube-api-access-648fx\") pod \"console-operator-85b454874b-6r9qv\" (UID: \"9c8d45b7-a264-43de-a9e8-3d8a4fb57669\") " pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.399662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.399090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf5vk\" (UniqueName: \"kubernetes.io/projected/ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b-kube-api-access-bf5vk\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dxjsb\" (UID: \"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.400256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.400015    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ddcg\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.401293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.401273    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pllbt\" (UniqueName: \"kubernetes.io/projected/a29dd7c6-d869-4497-9a3a-23a7289a6c58-kube-api-access-pllbt\") pod \"insights-operator-76bcd6ccf5-8rgkr\" (UID: \"a29dd7c6-d869-4497-9a3a-23a7289a6c58\") " pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.401386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.401344    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9s2\" (UniqueName: \"kubernetes.io/projected/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-kube-api-access-gv9s2\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.401690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.401667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zj5g\" (UniqueName: \"kubernetes.io/projected/af80a1bb-0779-48f4-9f78-9802c2015d70-kube-api-access-4zj5g\") pod \"service-ca-operator-859f94c4b-5mnjd\" (UID: \"af80a1bb-0779-48f4-9f78-9802c2015d70\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.403362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.402858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2wg7\" (UniqueName: \"kubernetes.io/projected/4c6da106-a66c-443a-8ee2-606a548948af-kube-api-access-h2wg7\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.437707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.437674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr"
Sep 10 19:16:16.461040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.460993    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"
Sep 10 19:16:16.470162 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.470134    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-skmtm"
Sep 10 19:16:16.477496 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.477470    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"
Sep 10 19:16:16.481874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.481848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgzw\" (UniqueName: \"kubernetes.io/projected/5d16ad53-c0ec-418c-8cdb-45db02f2e073-kube-api-access-lsgzw\") pod \"network-check-source-685456655f-8lvlx\" (UID: \"5d16ad53-c0ec-418c-8cdb-45db02f2e073\") " pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx"
Sep 10 19:16:16.491582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.491560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgzw\" (UniqueName: \"kubernetes.io/projected/5d16ad53-c0ec-418c-8cdb-45db02f2e073-kube-api-access-lsgzw\") pod \"network-check-source-685456655f-8lvlx\" (UID: \"5d16ad53-c0ec-418c-8cdb-45db02f2e073\") " pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx"
Sep 10 19:16:16.502650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.502627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:16.509292 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.509272    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx"
Sep 10 19:16:16.696573 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.696466    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:16.696757 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.696584    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:16.696757 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.696629    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:16.699793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.699396    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gqr5m\""
Sep 10 19:16:16.699793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.699414    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:16.699793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.699487    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8fmlh\""
Sep 10 19:16:16.699793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.699701    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:16.785319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.785283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:16.785512 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.785453    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:16.785586 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.785533    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.7855127 +0000 UTC m=+36.606568816 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:16.886390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.886574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:16.886574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886509    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:16.886574 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886527    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886502631 +0000 UTC m=+36.707558759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886607    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886612    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886628    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886663    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886649268 +0000 UTC m=+36.707705398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886712    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:16.886740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:16.886721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886751    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886737807 +0000 UTC m=+36.707793936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886780    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886770635 +0000 UTC m=+36.707826746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886796    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886807    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886849    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886835554 +0000 UTC m=+36.707891668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:16.887104 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:16.886871    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:17.886858641 +0000 UTC m=+36.707914752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:17.796789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.796540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:17.797259 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.796678    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:17.797259 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.796926    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.79690824 +0000 UTC m=+38.617964354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:17.897923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.897845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:17.897923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.897898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.897928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898008    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.898038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898064    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898074    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898056184 +0000 UTC m=+38.719112313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898125    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:17.898148 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898137    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.898159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898173    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898159696 +0000 UTC m=+38.719215810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898191    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898182974 +0000 UTC m=+38.719239085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898208    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898199892 +0000 UTC m=+38.719256003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898220    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898263    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898251337 +0000 UTC m=+38.719307453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.898293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:17.898404 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898386    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:17.898675 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:17.898415    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:19.898408119 +0000 UTC m=+38.719464229 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:17.956581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.956549    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb"]
Sep 10 19:16:17.957478 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.957459    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs"]
Sep 10 19:16:17.960599 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.960564    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2d4d25_6916_4e33_81f8_dd4a3f0ba92b.slice/crio-a352d83fc9090aa56c58ad1e366652a32a78726e63f4a40f26bea39a180c031f WatchSource:0}: Error finding container a352d83fc9090aa56c58ad1e366652a32a78726e63f4a40f26bea39a180c031f: Status 404 returned error can't find the container with id a352d83fc9090aa56c58ad1e366652a32a78726e63f4a40f26bea39a180c031f
Sep 10 19:16:17.961831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.961807    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-8lvlx"]
Sep 10 19:16:17.963787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.963280    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd"]
Sep 10 19:16:17.964846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.964801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-8rgkr"]
Sep 10 19:16:17.965822 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.965768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-6r9qv"]
Sep 10 19:16:17.967445 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.967411    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d16ad53_c0ec_418c_8cdb_45db02f2e073.slice/crio-fa9fa9a42f3fd510c24ade9fbbb5eb3090f291da4f801452482599a1f86d9670 WatchSource:0}: Error finding container fa9fa9a42f3fd510c24ade9fbbb5eb3090f291da4f801452482599a1f86d9670: Status 404 returned error can't find the container with id fa9fa9a42f3fd510c24ade9fbbb5eb3090f291da4f801452482599a1f86d9670
Sep 10 19:16:17.968506 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.968478    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf80a1bb_0779_48f4_9f78_9802c2015d70.slice/crio-994f188a74a1236840bfe90f2a1bfbfb8eedb2e890a97309970b117d9d3029e7 WatchSource:0}: Error finding container 994f188a74a1236840bfe90f2a1bfbfb8eedb2e890a97309970b117d9d3029e7: Status 404 returned error can't find the container with id 994f188a74a1236840bfe90f2a1bfbfb8eedb2e890a97309970b117d9d3029e7
Sep 10 19:16:17.971455 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.970959    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29dd7c6_d869_4497_9a3a_23a7289a6c58.slice/crio-f3644b2748c41f6e338a9878253e2d7ece7e85c470e10ce5808502c8afbeedb5 WatchSource:0}: Error finding container f3644b2748c41f6e338a9878253e2d7ece7e85c470e10ce5808502c8afbeedb5: Status 404 returned error can't find the container with id f3644b2748c41f6e338a9878253e2d7ece7e85c470e10ce5808502c8afbeedb5
Sep 10 19:16:17.972388 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.971645    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8d45b7_a264_43de_a9e8_3d8a4fb57669.slice/crio-1c641d3495f1cdf01579a13cdd88879c2b531c79974b15bdc99db70a9f3ce51f WatchSource:0}: Error finding container 1c641d3495f1cdf01579a13cdd88879c2b531c79974b15bdc99db70a9f3ce51f: Status 404 returned error can't find the container with id 1c641d3495f1cdf01579a13cdd88879c2b531c79974b15bdc99db70a9f3ce51f
Sep 10 19:16:17.983361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:17.983338    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-skmtm"]
Sep 10 19:16:17.986335 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:17.986314    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5bf6c7c_9f53_4bf9_ab7f_21e695929503.slice/crio-aedf366ed2280b6c16a8df37e2ff8134e259144d75b889db096f3d3d00e27afd WatchSource:0}: Error finding container aedf366ed2280b6c16a8df37e2ff8134e259144d75b889db096f3d3d00e27afd: Status 404 returned error can't find the container with id aedf366ed2280b6c16a8df37e2ff8134e259144d75b889db096f3d3d00e27afd
Sep 10 19:16:18.845514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.845476    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs" event={"ID":"ae7517b3-e699-4658-8e2a-919d98953038","Type":"ContainerStarted","Data":"7703996e612530e6f97ad78f6367c0cde64bbb8f8bdeaa0987d50ccee721b01f"}
Sep 10 19:16:18.847732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.847688    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd" event={"ID":"af80a1bb-0779-48f4-9f78-9802c2015d70","Type":"ContainerStarted","Data":"994f188a74a1236840bfe90f2a1bfbfb8eedb2e890a97309970b117d9d3029e7"}
Sep 10 19:16:18.849101 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.849072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-skmtm" event={"ID":"e5bf6c7c-9f53-4bf9-ab7f-21e695929503","Type":"ContainerStarted","Data":"aedf366ed2280b6c16a8df37e2ff8134e259144d75b889db096f3d3d00e27afd"}
Sep 10 19:16:18.850539 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.850510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" event={"ID":"9c8d45b7-a264-43de-a9e8-3d8a4fb57669","Type":"ContainerStarted","Data":"1c641d3495f1cdf01579a13cdd88879c2b531c79974b15bdc99db70a9f3ce51f"}
Sep 10 19:16:18.851994 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.851968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr" event={"ID":"a29dd7c6-d869-4497-9a3a-23a7289a6c58","Type":"ContainerStarted","Data":"f3644b2748c41f6e338a9878253e2d7ece7e85c470e10ce5808502c8afbeedb5"}
Sep 10 19:16:18.856491 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.856455    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="3018928f581eebdb9ac4a2790a18608c7b6ce459714a025c83940193cac559dd" exitCode=0
Sep 10 19:16:18.856589 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.856538    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"3018928f581eebdb9ac4a2790a18608c7b6ce459714a025c83940193cac559dd"}
Sep 10 19:16:18.858660 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.858634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb" event={"ID":"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b","Type":"ContainerStarted","Data":"a352d83fc9090aa56c58ad1e366652a32a78726e63f4a40f26bea39a180c031f"}
Sep 10 19:16:18.860505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:18.860114    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx" event={"ID":"5d16ad53-c0ec-418c-8cdb-45db02f2e073","Type":"ContainerStarted","Data":"fa9fa9a42f3fd510c24ade9fbbb5eb3090f291da4f801452482599a1f86d9670"}
Sep 10 19:16:19.819780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.818914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:19.819780 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.819348    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:19.819780 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.819416    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.819396692 +0000 UTC m=+42.640452826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:19.870467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.870389    2567 generic.go:356] "Generic (PLEG): container finished" podID="5817f7ec-d60b-4f3c-9e24-8aa71b716536" containerID="78d8c3e7c35de74641e47d0e2f963819ee8c02ca97adf982db1950589e813f09" exitCode=0
Sep 10 19:16:19.870467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.870464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerDied","Data":"78d8c3e7c35de74641e47d0e2f963819ee8c02ca97adf982db1950589e813f09"}
Sep 10 19:16:19.920026 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.919982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:19.920210 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.920048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:19.920210 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.920130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:19.920210 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.920164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:19.920210 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.920201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:19.920425 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:19.920244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:19.920479 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.920426    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:19.920530 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.920492    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:19.920530 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.920511    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.920656    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921009    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.920428    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921333    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.920475345 +0000 UTC m=+42.741531470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921361    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.921347275 +0000 UTC m=+42.742403389 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921388    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.92137859 +0000 UTC m=+42.742434776 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921404    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.921396562 +0000 UTC m=+42.742452672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921425    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.921416763 +0000 UTC m=+42.742472876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:19.921634 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:19.921438    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:23.921431126 +0000 UTC m=+42.742487240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:23.858303 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.858258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:23.858748 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.858386    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:23.858748 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.858461    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.858442601 +0000 UTC m=+50.679498724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:23.958900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.958859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:23.959110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.958963    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:23.959110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.958995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:23.959110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.959019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:23.959110 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959038    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:23.959110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.959051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959131    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:23.959144    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959149    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959195    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.959174388 +0000 UTC m=+50.780230520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959131    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959219    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.959203915 +0000 UTC m=+50.780260030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959235    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959242    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.959230715 +0000 UTC m=+50.780286828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959245    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959255    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.95924921 +0000 UTC m=+50.780305320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959264    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.959259706 +0000 UTC m=+50.780315821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:23.959342 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:23.959275    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.959270273 +0000 UTC m=+50.780326384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:26.895680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.895642    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/0.log"
Sep 10 19:16:26.896274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.895694    2567 generic.go:356] "Generic (PLEG): container finished" podID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669" containerID="3dda1a931d738820ffbc31dd5729bc6574b87f055465c6759ffb109c51992923" exitCode=255
Sep 10 19:16:26.896274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.895786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" event={"ID":"9c8d45b7-a264-43de-a9e8-3d8a4fb57669","Type":"ContainerDied","Data":"3dda1a931d738820ffbc31dd5729bc6574b87f055465c6759ffb109c51992923"}
Sep 10 19:16:26.896274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.896066    2567 scope.go:117] "RemoveContainer" containerID="3dda1a931d738820ffbc31dd5729bc6574b87f055465c6759ffb109c51992923"
Sep 10 19:16:26.899323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.898731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr" event={"ID":"a29dd7c6-d869-4497-9a3a-23a7289a6c58","Type":"ContainerStarted","Data":"9bb5df8ed51f7cfa93f2278c00940f03d69079a76cc3b398054858c553b0c561"}
Sep 10 19:16:26.909983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.909771    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" event={"ID":"5817f7ec-d60b-4f3c-9e24-8aa71b716536","Type":"ContainerStarted","Data":"5bfad30431efee03ed425e4b0eb5cb837117ff7c71f5b07abfcc2b37596eb3be"}
Sep 10 19:16:26.912756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.912308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb" event={"ID":"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b","Type":"ContainerStarted","Data":"4bb78035e365c794b21126a74bc18bd593bdda9a686af946df5ffc3266a9d7e0"}
Sep 10 19:16:26.915005 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.914582    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx" event={"ID":"5d16ad53-c0ec-418c-8cdb-45db02f2e073","Type":"ContainerStarted","Data":"90bda355d40ff7a4ab9e71d08c7a4f5701e9d39cb2904731e18028932f81e1e4"}
Sep 10 19:16:26.919021 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.918500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs" event={"ID":"ae7517b3-e699-4658-8e2a-919d98953038","Type":"ContainerStarted","Data":"b7f13483fa8e552353823d0e64b25169eea0ebacd9f987a300304cadf0a22d84"}
Sep 10 19:16:26.921344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.920839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd" event={"ID":"af80a1bb-0779-48f4-9f78-9802c2015d70","Type":"ContainerStarted","Data":"04e25820b6e25086b19574d49911d6927b486a2e810c904f3a029eeee76905cc"}
Sep 10 19:16:26.923507 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.923485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-skmtm" event={"ID":"e5bf6c7c-9f53-4bf9-ab7f-21e695929503","Type":"ContainerStarted","Data":"65eec589d6a57f60151020df476aab2ccda35e26c85de60c729da29e2e5eeb93"}
Sep 10 19:16:26.943921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.943858    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wxv8f" podStartSLOduration=13.264770687 podStartE2EDuration="44.943835703s" totalImagesPullingTime="31.679065016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:26.941984106 +0000 UTC m=+45.763040239" watchObservedRunningTime="2026-09-10 19:16:26.943835703 +0000 UTC m=+45.764891836"
Sep 10 19:16:26.979379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.979317    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr" podStartSLOduration=29.005627776 podStartE2EDuration="36.979298524s" totalImagesPullingTime="7.973670748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:26.959652346 +0000 UTC m=+45.780708505" watchObservedRunningTime="2026-09-10 19:16:26.979298524 +0000 UTC m=+45.800354656"
Sep 10 19:16:26.979565 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.979512    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb" podStartSLOduration=28.998105713 podStartE2EDuration="36.97950644s" totalImagesPullingTime="7.981400727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:26.978589622 +0000 UTC m=+45.799645755" watchObservedRunningTime="2026-09-10 19:16:26.97950644 +0000 UTC m=+45.800562573"
Sep 10 19:16:26.995818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:26.995559    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-8lvlx" podStartSLOduration=29.006141449 podStartE2EDuration="36.995535874s" totalImagesPullingTime="7.989394425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:26.994498681 +0000 UTC m=+45.815554814" watchObservedRunningTime="2026-09-10 19:16:26.995535874 +0000 UTC m=+45.816592008"
Sep 10 19:16:27.018958 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.018768    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-skmtm" podStartSLOduration=3.057607174 podStartE2EDuration="11.018742418s" totalImagesPullingTime="7.961135244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:27.016218871 +0000 UTC m=+45.837275005" watchObservedRunningTime="2026-09-10 19:16:27.018742418 +0000 UTC m=+45.839798552"
Sep 10 19:16:27.040996 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.040919    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-rpwzs" podStartSLOduration=29.103614545 podStartE2EDuration="37.040897263s" totalImagesPullingTime="7.937282718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:27.039193675 +0000 UTC m=+45.860249814" watchObservedRunningTime="2026-09-10 19:16:27.040897263 +0000 UTC m=+45.861953401"
Sep 10 19:16:27.615233 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.615168    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd" podStartSLOduration=29.638102252 podStartE2EDuration="37.61513976s" totalImagesPullingTime="7.977037508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:27.062594135 +0000 UTC m=+45.883650268" watchObservedRunningTime="2026-09-10 19:16:27.61513976 +0000 UTC m=+46.436195895"
Sep 10 19:16:27.615797 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.615766    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zch66"]
Sep 10 19:16:27.639497 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.639467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zch66"]
Sep 10 19:16:27.639655 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.639583    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.641831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.641804    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 19:16:27.641980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.641835    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 19:16:27.642095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.642082    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mhsqp\""
Sep 10 19:16:27.697050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.697015    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr4rc\" (UniqueName: \"kubernetes.io/projected/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-api-access-pr4rc\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.697239 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.697149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.697239 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.697169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38bd8fb3-555c-4b1e-9ee7-6663278dd749-data-volume\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.697239 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.697188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38bd8fb3-555c-4b1e-9ee7-6663278dd749-crio-socket\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.697397 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.697317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798167 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798167 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38bd8fb3-555c-4b1e-9ee7-6663278dd749-data-volume\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798403 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38bd8fb3-555c-4b1e-9ee7-6663278dd749-crio-socket\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798403 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:27.798322    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:27.798403 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/38bd8fb3-555c-4b1e-9ee7-6663278dd749-crio-socket\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798512 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:27.798419    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.298397335 +0000 UTC m=+47.119453451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:27.798512 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:27.798446    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:27.798512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr4rc\" (UniqueName: \"kubernetes.io/projected/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-api-access-pr4rc\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.798512 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:27.798512    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:28.298498277 +0000 UTC m=+47.119554405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:27.798788 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.798553    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/38bd8fb3-555c-4b1e-9ee7-6663278dd749-data-volume\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.811035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.811001    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr4rc\" (UniqueName: \"kubernetes.io/projected/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-api-access-pr4rc\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:27.930281 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.930243    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:16:27.930719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.930698    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/0.log"
Sep 10 19:16:27.930764 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.930739    2567 generic.go:356] "Generic (PLEG): container finished" podID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669" containerID="fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1" exitCode=255
Sep 10 19:16:27.930844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.930824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" event={"ID":"9c8d45b7-a264-43de-a9e8-3d8a4fb57669","Type":"ContainerDied","Data":"fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1"}
Sep 10 19:16:27.930896 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.930879    2567 scope.go:117] "RemoveContainer" containerID="3dda1a931d738820ffbc31dd5729bc6574b87f055465c6759ffb109c51992923"
Sep 10 19:16:27.931201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:27.931170    2567 scope.go:117] "RemoveContainer" containerID="fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1"
Sep 10 19:16:27.931382 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:27.931360    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-6r9qv_openshift-console-operator(9c8d45b7-a264-43de-a9e8-3d8a4fb57669)\"" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" podUID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669"
Sep 10 19:16:28.304159 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:28.304077    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:28.304316 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:28.304211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:28.304316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:28.304231    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:28.304316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:28.304308    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:28.304316 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:28.304312    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:29.30428913 +0000 UTC m=+48.125345240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:28.304455 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:28.304367    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:29.304345978 +0000 UTC m=+48.125402089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:28.936214 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:28.936185    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:16:28.936623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:28.936535    2567 scope.go:117] "RemoveContainer" containerID="fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1"
Sep 10 19:16:28.936713 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:28.936695    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-6r9qv_openshift-console-operator(9c8d45b7-a264-43de-a9e8-3d8a4fb57669)\"" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" podUID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669"
Sep 10 19:16:29.313287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:29.313194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:29.313287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:29.313272    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:29.313491 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:29.313331    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:29.313491 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:29.313357    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:29.313491 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:29.313402    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.31338637 +0000 UTC m=+50.134442485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:29.313491 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:29.313416    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:31.313410088 +0000 UTC m=+50.134466200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:29.379910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:29.379878    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ck8rf_df856401-4913-4059-9bce-5bdf0655c1bd/dns-node-resolver/0.log"
Sep 10 19:16:30.781698 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:30.781669    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lbmvr_264a5601-4ce3-4a62-9baf-0f9c175a4244/node-ca/0.log"
Sep 10 19:16:31.178313 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:31.178286    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-skmtm_e5bf6c7c-9f53-4bf9-ab7f-21e695929503/serve-healthcheck-canary/0.log"
Sep 10 19:16:31.331434 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:31.331393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:31.331640 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:31.331472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:31.331640 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.331525    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 19:16:31.331640 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.331563    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:31.331640 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.331604    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.331583474 +0000 UTC m=+54.152639584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : configmap "kube-rbac-proxy" not found
Sep 10 19:16:31.331640 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.331623    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:35.33161461 +0000 UTC m=+54.152670724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:31.936647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:31.936600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:31.937085 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.936753    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:31.937085 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:31.936821    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls podName:5a084ee9-b2eb-4e9a-ae5d-45957770d2c4 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:47.936806302 +0000 UTC m=+66.757862413 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-64ktg" (UID: "5a084ee9-b2eb-4e9a-ae5d-45957770d2c4") : secret "cluster-monitoring-operator-tls" not found
Sep 10 19:16:32.037626 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037750    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037764    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037750    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 19:16:32.037828 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037823    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.037802313 +0000 UTC m=+66.858858442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : secret "router-metrics-certs-default" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037847    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls podName:4c6da106-a66c-443a-8ee2-606a548948af nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.037838378 +0000 UTC m=+66.858894489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-2mmnk" (UID: "4c6da106-a66c-443a-8ee2-606a548948af") : secret "samples-operator-tls" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037861    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037864    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert podName:6e6dbc24-2f34-48d9-9233-26f2ab31da70 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.037855467 +0000 UTC m=+66.858911581 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mr9td" (UID: "6e6dbc24-2f34-48d9-9233-26f2ab31da70") : secret "networking-console-plugin-cert" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037873    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx: secret "image-registry-tls" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037884    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle podName:cb45bf43-4568-4efe-8593-2c61a1950857 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.037876187 +0000 UTC m=+66.858932299 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle") pod "router-default-cff896496-rh6zc" (UID: "cb45bf43-4568-4efe-8593-2c61a1950857") : configmap references non-existent config key: service-ca.crt
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.037919    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls podName:3a29c116-bcd1-4641-b3b1-5ddeadc1e839 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.037903243 +0000 UTC m=+66.858959357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls") pod "image-registry-5b6cf6b8f4-6hbdx" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839") : secret "image-registry-tls" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:32.037975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.038034    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 19:16:32.038221 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:32.038079    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls podName:e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:48.038066893 +0000 UTC m=+66.859123009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls") pod "dns-default-psnrk" (UID: "e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2") : secret "dns-default-metrics-tls" not found
Sep 10 19:16:35.367081 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:35.367039    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:35.367547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:35.367119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:35.367547 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:35.367230    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:35.367547 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:35.367284    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls podName:38bd8fb3-555c-4b1e-9ee7-6663278dd749 nodeName:}" failed. No retries permitted until 2026-09-10 19:16:43.367271579 +0000 UTC m=+62.188327691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zch66" (UID: "38bd8fb3-555c-4b1e-9ee7-6663278dd749") : secret "insights-runtime-extractor-tls" not found
Sep 10 19:16:35.367681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:35.367650    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/38bd8fb3-555c-4b1e-9ee7-6663278dd749-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:36.503434 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:36.503392    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:36.503824 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:36.503457    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:36.503870 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:36.503837    2567 scope.go:117] "RemoveContainer" containerID="fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1"
Sep 10 19:16:36.504030 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:16:36.504012    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-6r9qv_openshift-console-operator(9c8d45b7-a264-43de-a9e8-3d8a4fb57669)\"" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" podUID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669"
Sep 10 19:16:40.839070 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:40.839038    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r7vb4"
Sep 10 19:16:43.439335 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.439290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:43.441501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.441475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/38bd8fb3-555c-4b1e-9ee7-6663278dd749-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zch66\" (UID: \"38bd8fb3-555c-4b1e-9ee7-6663278dd749\") " pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:43.551667 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.551633    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mhsqp\""
Sep 10 19:16:43.560347 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.560324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zch66"
Sep 10 19:16:43.832269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.832195    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zch66"]
Sep 10 19:16:43.835093 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:43.835068    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bd8fb3_555c_4b1e_9ee7_6663278dd749.slice/crio-5cb25378bd32ffb2058d21e3e24e94931ddb878626d4ce00df995c58ee1d3360 WatchSource:0}: Error finding container 5cb25378bd32ffb2058d21e3e24e94931ddb878626d4ce00df995c58ee1d3360: Status 404 returned error can't find the container with id 5cb25378bd32ffb2058d21e3e24e94931ddb878626d4ce00df995c58ee1d3360
Sep 10 19:16:43.986719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.986687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zch66" event={"ID":"38bd8fb3-555c-4b1e-9ee7-6663278dd749","Type":"ContainerStarted","Data":"41e89f4991b26eb6bdc2744b5bea100270dc1798e365eabcb18386d34f09cac2"}
Sep 10 19:16:43.986879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:43.986741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zch66" event={"ID":"38bd8fb3-555c-4b1e-9ee7-6663278dd749","Type":"ContainerStarted","Data":"5cb25378bd32ffb2058d21e3e24e94931ddb878626d4ce00df995c58ee1d3360"}
Sep 10 19:16:44.991461 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:44.991375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zch66" event={"ID":"38bd8fb3-555c-4b1e-9ee7-6663278dd749","Type":"ContainerStarted","Data":"d31b9563c0d7e13c316dd2e13d0695139687dfaf428b4e22b3550875b3828f2b"}
Sep 10 19:16:47.000187 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.000149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zch66" event={"ID":"38bd8fb3-555c-4b1e-9ee7-6663278dd749","Type":"ContainerStarted","Data":"810895296d63aa634676416a78a74d4afe6353d71991ad3943dc3f510fdc1cd6"}
Sep 10 19:16:47.027121 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.027073    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zch66" podStartSLOduration=17.859623696 podStartE2EDuration="20.027056608s" totalImagesPullingTime="2.167432912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:16:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:47.025346222 +0000 UTC m=+65.846402354" watchObservedRunningTime="2026-09-10 19:16:47.027056608 +0000 UTC m=+65.848112742"
Sep 10 19:16:47.378766 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.378689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:47.381450 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.381418    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 19:16:47.391598 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.391570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed5274b-92ef-48e0-9466-a04023ae4f93-original-pull-secret\") pod \"global-pull-secret-syncer-n7lpb\" (UID: \"9ed5274b-92ef-48e0-9466-a04023ae4f93\") " pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:47.480111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.480074    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:47.482505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.482485    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 19:16:47.492879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.492852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f535af16-3f8b-43f9-9c39-59b1a86b9080-metrics-certs\") pod \"network-metrics-daemon-6qj78\" (UID: \"f535af16-3f8b-43f9-9c39-59b1a86b9080\") " pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:47.581094 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.581049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:47.583551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.583528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wz7p\" (UniqueName: \"kubernetes.io/projected/203eda7c-4e7a-42cc-b225-ef360158a40b-kube-api-access-9wz7p\") pod \"network-check-target-4qdbk\" (UID: \"203eda7c-4e7a-42cc-b225-ef360158a40b\") " pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:47.612334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.612308    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8fmlh\""
Sep 10 19:16:47.616656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.616641    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7lpb"
Sep 10 19:16:47.619765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.619748    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:47.623907 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.623892    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gqr5m\""
Sep 10 19:16:47.631913 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.631895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6qj78"
Sep 10 19:16:47.919763 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.919732    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qdbk"]
Sep 10 19:16:47.922962 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.922868    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7lpb"]
Sep 10 19:16:47.923632 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:47.923612    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203eda7c_4e7a_42cc_b225_ef360158a40b.slice/crio-adb4fcda45a32bd2dfc9426c37689b0f302c7e17f652999b6560e88edc7c1754 WatchSource:0}: Error finding container adb4fcda45a32bd2dfc9426c37689b0f302c7e17f652999b6560e88edc7c1754: Status 404 returned error can't find the container with id adb4fcda45a32bd2dfc9426c37689b0f302c7e17f652999b6560e88edc7c1754
Sep 10 19:16:47.925133 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:47.925113    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed5274b_92ef_48e0_9466_a04023ae4f93.slice/crio-c84d451f4bb359756c3dd9ccb176ee5eb49629d3eb45bfa0c8b68ad195217b98 WatchSource:0}: Error finding container c84d451f4bb359756c3dd9ccb176ee5eb49629d3eb45bfa0c8b68ad195217b98: Status 404 returned error can't find the container with id c84d451f4bb359756c3dd9ccb176ee5eb49629d3eb45bfa0c8b68ad195217b98
Sep 10 19:16:47.940046 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.940019    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6qj78"]
Sep 10 19:16:47.943287 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:47.943265    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf535af16_3f8b_43f9_9c39_59b1a86b9080.slice/crio-7b9f8b5760acc11636de5f06b09d2517152ed3235b79e52dfd1d340509c72d73 WatchSource:0}: Error finding container 7b9f8b5760acc11636de5f06b09d2517152ed3235b79e52dfd1d340509c72d73: Status 404 returned error can't find the container with id 7b9f8b5760acc11636de5f06b09d2517152ed3235b79e52dfd1d340509c72d73
Sep 10 19:16:47.983285 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.983255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:47.985857 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:47.985836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a084ee9-b2eb-4e9a-ae5d-45957770d2c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-64ktg\" (UID: \"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:48.004631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.004599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qdbk" event={"ID":"203eda7c-4e7a-42cc-b225-ef360158a40b","Type":"ContainerStarted","Data":"adb4fcda45a32bd2dfc9426c37689b0f302c7e17f652999b6560e88edc7c1754"}
Sep 10 19:16:48.005770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.005744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7lpb" event={"ID":"9ed5274b-92ef-48e0-9466-a04023ae4f93","Type":"ContainerStarted","Data":"c84d451f4bb359756c3dd9ccb176ee5eb49629d3eb45bfa0c8b68ad195217b98"}
Sep 10 19:16:48.006889 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.006867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6qj78" event={"ID":"f535af16-3f8b-43f9-9c39-59b1a86b9080","Type":"ContainerStarted","Data":"7b9f8b5760acc11636de5f06b09d2517152ed3235b79e52dfd1d340509c72d73"}
Sep 10 19:16:48.083818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.083784    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:48.083818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.083830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:48.084112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.083877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:48.084112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.083908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:48.084112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.084087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:48.084284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.084138    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:48.084923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.084894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb45bf43-4568-4efe-8593-2c61a1950857-service-ca-bundle\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:48.086470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.086440    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2-metrics-tls\") pod \"dns-default-psnrk\" (UID: \"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2\") " pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:48.086564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.086505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6e6dbc24-2f34-48d9-9233-26f2ab31da70-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mr9td\" (UID: \"6e6dbc24-2f34-48d9-9233-26f2ab31da70\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:48.086564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.086555    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4c6da106-a66c-443a-8ee2-606a548948af-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-2mmnk\" (UID: \"4c6da106-a66c-443a-8ee2-606a548948af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:48.086862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.086839    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"image-registry-5b6cf6b8f4-6hbdx\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") " pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:48.086905 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.086839    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb45bf43-4568-4efe-8593-2c61a1950857-metrics-certs\") pod \"router-default-cff896496-rh6zc\" (UID: \"cb45bf43-4568-4efe-8593-2c61a1950857\") " pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:48.200818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.200742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pbp9b\""
Sep 10 19:16:48.208477 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.208461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"
Sep 10 19:16:48.223334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.223312    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qxq2w\""
Sep 10 19:16:48.231795 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.231771    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:48.233293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.233277    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qr7g6\""
Sep 10 19:16:48.241450 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.241431    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"
Sep 10 19:16:48.247516 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.247495    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wkm4c\""
Sep 10 19:16:48.254775 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.254742    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:48.255942 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.255915    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rklhx\""
Sep 10 19:16:48.263912 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.263892    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:48.298037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.298004    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zv8kr\""
Sep 10 19:16:48.305724 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.305660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td"
Sep 10 19:16:48.501375 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.501299    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg"]
Sep 10 19:16:48.504893 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:48.504861    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a084ee9_b2eb_4e9a_ae5d_45957770d2c4.slice/crio-4cff4c981b653311f7a5f4b273cda6f7879232a118f6336b7e8ee4ee13f14f7f WatchSource:0}: Error finding container 4cff4c981b653311f7a5f4b273cda6f7879232a118f6336b7e8ee4ee13f14f7f: Status 404 returned error can't find the container with id 4cff4c981b653311f7a5f4b273cda6f7879232a118f6336b7e8ee4ee13f14f7f
Sep 10 19:16:48.536400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.536362    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:16:48.539389 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:48.539362    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a29c116_bcd1_4641_b3b1_5ddeadc1e839.slice/crio-61583ff6d8af29a693b0b6ef102bae8a8efa690cd63ce32c2a3bfacdb9f99cbd WatchSource:0}: Error finding container 61583ff6d8af29a693b0b6ef102bae8a8efa690cd63ce32c2a3bfacdb9f99cbd: Status 404 returned error can't find the container with id 61583ff6d8af29a693b0b6ef102bae8a8efa690cd63ce32c2a3bfacdb9f99cbd
Sep 10 19:16:48.562410 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.562381    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-psnrk"]
Sep 10 19:16:48.565527 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:48.565504    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e23b75_a206_4a9f_9fe6_e2d861a3a2d2.slice/crio-0f1f7f8d607b23429e72b698d6c84c6d36f3d90c8657c91855a990115f37f528 WatchSource:0}: Error finding container 0f1f7f8d607b23429e72b698d6c84c6d36f3d90c8657c91855a990115f37f528: Status 404 returned error can't find the container with id 0f1f7f8d607b23429e72b698d6c84c6d36f3d90c8657c91855a990115f37f528
Sep 10 19:16:48.590673 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.590646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cff896496-rh6zc"]
Sep 10 19:16:48.600369 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:48.600338    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb45bf43_4568_4efe_8593_2c61a1950857.slice/crio-b8568736d1575817900c4aa85b93e6cb23811532b3e22e0e351cd310834625b6 WatchSource:0}: Error finding container b8568736d1575817900c4aa85b93e6cb23811532b3e22e0e351cd310834625b6: Status 404 returned error can't find the container with id b8568736d1575817900c4aa85b93e6cb23811532b3e22e0e351cd310834625b6
Sep 10 19:16:48.618619 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.618432    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk"]
Sep 10 19:16:48.650701 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:48.650673    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mr9td"]
Sep 10 19:16:48.654879 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:48.654837    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6dbc24_2f34_48d9_9233_26f2ab31da70.slice/crio-fea1a32d9bf2f4dd7fb6cd5336d72e2b98e09191afe3ecba2b0640f94021e0fc WatchSource:0}: Error finding container fea1a32d9bf2f4dd7fb6cd5336d72e2b98e09191afe3ecba2b0640f94021e0fc: Status 404 returned error can't find the container with id fea1a32d9bf2f4dd7fb6cd5336d72e2b98e09191afe3ecba2b0640f94021e0fc
Sep 10 19:16:49.013181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.013141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td" event={"ID":"6e6dbc24-2f34-48d9-9233-26f2ab31da70","Type":"ContainerStarted","Data":"fea1a32d9bf2f4dd7fb6cd5336d72e2b98e09191afe3ecba2b0640f94021e0fc"}
Sep 10 19:16:49.015439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.015411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" event={"ID":"3a29c116-bcd1-4641-b3b1-5ddeadc1e839","Type":"ContainerStarted","Data":"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"}
Sep 10 19:16:49.015564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.015458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" event={"ID":"3a29c116-bcd1-4641-b3b1-5ddeadc1e839","Type":"ContainerStarted","Data":"61583ff6d8af29a693b0b6ef102bae8a8efa690cd63ce32c2a3bfacdb9f99cbd"}
Sep 10 19:16:49.015634 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.015589    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:16:49.017139 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.017111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg" event={"ID":"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4","Type":"ContainerStarted","Data":"4cff4c981b653311f7a5f4b273cda6f7879232a118f6336b7e8ee4ee13f14f7f"}
Sep 10 19:16:49.019167 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.019122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qdbk" event={"ID":"203eda7c-4e7a-42cc-b225-ef360158a40b","Type":"ContainerStarted","Data":"53a6b623cf2fc3375e3b5ccc96bcdd11cb7411a0c7eb8d65cc4c7d5dbf562536"}
Sep 10 19:16:49.019300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.019278    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:16:49.020719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.020693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk" event={"ID":"4c6da106-a66c-443a-8ee2-606a548948af","Type":"ContainerStarted","Data":"67981bacc8d4851519b91d590e250bd83aa31392cef4e511e8e36ef0fd845028"}
Sep 10 19:16:49.023460 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.023432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff896496-rh6zc" event={"ID":"cb45bf43-4568-4efe-8593-2c61a1950857","Type":"ContainerStarted","Data":"3f91b27dc4b991e510e21a6f66804c6cdb0dad011112794d410e04dd9c0351fa"}
Sep 10 19:16:49.023576 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.023478    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cff896496-rh6zc" event={"ID":"cb45bf43-4568-4efe-8593-2c61a1950857","Type":"ContainerStarted","Data":"b8568736d1575817900c4aa85b93e6cb23811532b3e22e0e351cd310834625b6"}
Sep 10 19:16:49.025290 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.025264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psnrk" event={"ID":"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2","Type":"ContainerStarted","Data":"0f1f7f8d607b23429e72b698d6c84c6d36f3d90c8657c91855a990115f37f528"}
Sep 10 19:16:49.060234 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.059293    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4qdbk" podStartSLOduration=67.059273503 podStartE2EDuration="1m7.059273503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:49.058883025 +0000 UTC m=+67.879939159" watchObservedRunningTime="2026-09-10 19:16:49.059273503 +0000 UTC m=+67.880329637"
Sep 10 19:16:49.060234 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.060050    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" podStartSLOduration=66.060035844 podStartE2EDuration="1m6.060035844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:49.040069105 +0000 UTC m=+67.861125241" watchObservedRunningTime="2026-09-10 19:16:49.060035844 +0000 UTC m=+67.881091981"
Sep 10 19:16:49.077718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.077640    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-cff896496-rh6zc" podStartSLOduration=59.077625782 podStartE2EDuration="59.077625782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:49.077245552 +0000 UTC m=+67.898301688" watchObservedRunningTime="2026-09-10 19:16:49.077625782 +0000 UTC m=+67.898681915"
Sep 10 19:16:49.264324 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.264236    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:49.267894 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.267776    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:49.698773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:49.696613    2567 scope.go:117] "RemoveContainer" containerID="fea5776e96cac72f4059fcff658e24d9f64b2c115e1ae4460c2c2d52f34c02d1"
Sep 10 19:16:50.049416 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.049333    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:16:50.049857 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.049442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" event={"ID":"9c8d45b7-a264-43de-a9e8-3d8a4fb57669","Type":"ContainerStarted","Data":"005a9be21e5fdd26131a375e2073a4fb410dac7a15ede662c177464e8d93e16e"}
Sep 10 19:16:50.050389 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.050191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:50.056330 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.055097    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6qj78" event={"ID":"f535af16-3f8b-43f9-9c39-59b1a86b9080","Type":"ContainerStarted","Data":"2b9460841e2daf2487f287a038aa3c094d640a4a8ba8cee120bdf11639667b20"}
Sep 10 19:16:50.057236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.057069    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:50.058590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.058545    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-cff896496-rh6zc"
Sep 10 19:16:50.091266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:50.090220    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" podStartSLOduration=52.159546274 podStartE2EDuration="1m0.090198407s" totalImagesPullingTime="7.930652133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:50.069033182 +0000 UTC m=+68.890089316" watchObservedRunningTime="2026-09-10 19:16:50.090198407 +0000 UTC m=+68.911254541"
Sep 10 19:16:51.050665 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:51.050612    2567 patch_prober.go:28] interesting pod/console-operator-85b454874b-6r9qv container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.17:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:16:51.051123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:51.050696    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-6r9qv" podUID="9c8d45b7-a264-43de-a9e8-3d8a4fb57669" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.17:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:16:51.084695 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:51.084664    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-6r9qv"
Sep 10 19:16:54.072955 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.072849    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7lpb" event={"ID":"9ed5274b-92ef-48e0-9466-a04023ae4f93","Type":"ContainerStarted","Data":"d31fa6b6cb5a03da847fecc66f7d75b064a9fe092d7e160b4eab169a5d818189"}
Sep 10 19:16:54.074685 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.074659    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk" event={"ID":"4c6da106-a66c-443a-8ee2-606a548948af","Type":"ContainerStarted","Data":"5d2e4c7c4c527055f60e3254d69ac2d5e76c16f420b646dd156224e1e67a2adc"}
Sep 10 19:16:54.074816 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.074708    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk" event={"ID":"4c6da106-a66c-443a-8ee2-606a548948af","Type":"ContainerStarted","Data":"9bd89d0332462c0815314153bdc8db70a4d146fa34fea51e0a6ed1cca1b71317"}
Sep 10 19:16:54.076620 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.076598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psnrk" event={"ID":"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2","Type":"ContainerStarted","Data":"6cbbf0ea7a42f77bd9316eeae935a04d67c084a64934557571eb9c619df49762"}
Sep 10 19:16:54.076715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.076645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-psnrk" event={"ID":"e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2","Type":"ContainerStarted","Data":"0ef499d34fc7f26810c2210b218b6a22233e4537872a3ffaf81901eefbc0fc92"}
Sep 10 19:16:54.076785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.076710    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-psnrk"
Sep 10 19:16:54.078193 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.078174    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td" event={"ID":"6e6dbc24-2f34-48d9-9233-26f2ab31da70","Type":"ContainerStarted","Data":"0d875307572b794efa5863f404235a2ab186aeef70ad5a7b5aa2aac3de6f749b"}
Sep 10 19:16:54.079586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.079568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg" event={"ID":"5a084ee9-b2eb-4e9a-ae5d-45957770d2c4","Type":"ContainerStarted","Data":"509e448254eb0d7b2691c7fcafdd6106fb669664104833ff3eb8cfa12bae7e8f"}
Sep 10 19:16:54.081351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.081334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6qj78" event={"ID":"f535af16-3f8b-43f9-9c39-59b1a86b9080","Type":"ContainerStarted","Data":"84449813b3e6bec74d9218ba955b40e125dc6db83ac44f3154563b7851693d08"}
Sep 10 19:16:54.089696 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.089656    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n7lpb" podStartSLOduration=66.427866598 podStartE2EDuration="1m12.089643902s" totalImagesPullingTime="5.661777304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.089438221 +0000 UTC m=+72.910494355" watchObservedRunningTime="2026-09-10 19:16:54.089643902 +0000 UTC m=+72.910700034"
Sep 10 19:16:54.106057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.106016    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-mr9td" podStartSLOduration=41.231599772 podStartE2EDuration="46.106003024s" totalImagesPullingTime="4.874403252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.105588101 +0000 UTC m=+72.926644234" watchObservedRunningTime="2026-09-10 19:16:54.106003024 +0000 UTC m=+72.927059156"
Sep 10 19:16:54.129495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.129436    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-64ktg" podStartSLOduration=59.101549297 podStartE2EDuration="1m4.129415251s" totalImagesPullingTime="5.027865954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.12881648 +0000 UTC m=+72.949872614" watchObservedRunningTime="2026-09-10 19:16:54.129415251 +0000 UTC m=+72.950471381"
Sep 10 19:16:54.148642 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.148580    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6qj78" podStartSLOduration=70.336885056 podStartE2EDuration="1m12.14855894s" totalImagesPullingTime="1.811673884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.147111527 +0000 UTC m=+72.968167661" watchObservedRunningTime="2026-09-10 19:16:54.14855894 +0000 UTC m=+72.969615074"
Sep 10 19:16:54.171581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.171531    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-2mmnk" podStartSLOduration=59.322992904 podStartE2EDuration="1m4.17151416s" totalImagesPullingTime="4.848521256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.170427078 +0000 UTC m=+72.991483211" watchObservedRunningTime="2026-09-10 19:16:54.17151416 +0000 UTC m=+72.992570294"
Sep 10 19:16:54.191650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.191593    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-psnrk" podStartSLOduration=33.226290914 podStartE2EDuration="38.191577114s" totalImagesPullingTime="4.9652862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:54.18980733 +0000 UTC m=+73.010863462" watchObservedRunningTime="2026-09-10 19:16:54.191577114 +0000 UTC m=+73.012633247"
Sep 10 19:16:54.225846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.225817    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"]
Sep 10 19:16:54.229754 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.229733    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:54.232474 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.232450    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 19:16:54.232591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.232490    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8b4tg\""
Sep 10 19:16:54.247592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.247570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"]
Sep 10 19:16:54.342420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.342335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/51e9ada4-7ce2-4c09-afdd-5d3883faee76-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-xt6kj\" (UID: \"51e9ada4-7ce2-4c09-afdd-5d3883faee76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:54.443067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.443031    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/51e9ada4-7ce2-4c09-afdd-5d3883faee76-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-xt6kj\" (UID: \"51e9ada4-7ce2-4c09-afdd-5d3883faee76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:54.445449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.445428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/51e9ada4-7ce2-4c09-afdd-5d3883faee76-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-xt6kj\" (UID: \"51e9ada4-7ce2-4c09-afdd-5d3883faee76\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:54.538699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.538669    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:54.807915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:54.807821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"]
Sep 10 19:16:54.810375 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:16:54.810351    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e9ada4_7ce2_4c09_afdd_5d3883faee76.slice/crio-64582ed2cbc5fea575bce4c85efb9838a9b82591be6a7c0452feabc981793327 WatchSource:0}: Error finding container 64582ed2cbc5fea575bce4c85efb9838a9b82591be6a7c0452feabc981793327: Status 404 returned error can't find the container with id 64582ed2cbc5fea575bce4c85efb9838a9b82591be6a7c0452feabc981793327
Sep 10 19:16:55.087178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:55.087086    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj" event={"ID":"51e9ada4-7ce2-4c09-afdd-5d3883faee76","Type":"ContainerStarted","Data":"64582ed2cbc5fea575bce4c85efb9838a9b82591be6a7c0452feabc981793327"}
Sep 10 19:16:57.102622 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:57.102580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj" event={"ID":"51e9ada4-7ce2-4c09-afdd-5d3883faee76","Type":"ContainerStarted","Data":"8640c26930520e17ebba4d00015a3928f97b5377f684e64a9dbf5be40a6d792e"}
Sep 10 19:16:57.103031 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:57.102675    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:57.107578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:57.107556    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj"
Sep 10 19:16:57.121451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:16:57.121406    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-xt6kj" podStartSLOduration=1.394631915 podStartE2EDuration="3.121390995s" totalImagesPullingTime="1.72675908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:16:57.119495497 +0000 UTC m=+75.940551630" watchObservedRunningTime="2026-09-10 19:16:57.121390995 +0000 UTC m=+75.942447127"
Sep 10 19:17:02.659481 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.659445    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"]
Sep 10 19:17:02.662735 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.662718    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.665116 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.665094    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 19:17:02.665351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.665327    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5vw7n\""
Sep 10 19:17:02.665351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.665336    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 19:17:02.665998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.665983    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 19:17:02.674627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.674598    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"]
Sep 10 19:17:02.687428 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.687393    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-htcj2"]
Sep 10 19:17:02.691278 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.691258    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.693370 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.693351    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 19:17:02.693639 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.693620    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg7ls\""
Sep 10 19:17:02.693760 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.693679    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 19:17:02.693760 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.693704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 19:17:02.706616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706594    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.706716 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.706716 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjkr2\" (UniqueName: \"kubernetes.io/projected/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-kube-api-access-qjkr2\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.706849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-wtmp\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.706849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706753    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-textfile\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.706849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zqcf\" (UniqueName: \"kubernetes.io/projected/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-kube-api-access-2zqcf\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.706849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706825    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-root\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.706849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-metrics-client-ca\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.707038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.707038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706881    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-tls\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.707038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-sys\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.707038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-accelerators-collector-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.707038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.706982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.807577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.807541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.807750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.807594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.807750 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:02.807679    2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 19:17:02.807750 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:02.807732    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls podName:a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:03.307715845 +0000 UTC m=+82.128771956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-6cpxs" (UID: "a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823") : secret "openshift-state-metrics-tls" not found
Sep 10 19:17:02.807879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.807758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.807879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.807793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjkr2\" (UniqueName: \"kubernetes.io/projected/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-kube-api-access-qjkr2\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.807958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-wtmp\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-textfile\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zqcf\" (UniqueName: \"kubernetes.io/projected/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-kube-api-access-2zqcf\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.808140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-root\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-metrics-client-ca\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-wtmp\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-tls\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-sys\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-accelerators-collector-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-root\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-textfile\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808635 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-sys\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808877 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808851    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-metrics-client-ca\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.808986 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.808864    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-accelerators-collector-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.809066 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.809046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.810238 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.810216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.810303 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.810235    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.810577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.810548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-node-exporter-tls\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:02.819411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.819387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zqcf\" (UniqueName: \"kubernetes.io/projected/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-kube-api-access-2zqcf\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:02.819868 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:02.819854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjkr2\" (UniqueName: \"kubernetes.io/projected/9fdfcb26-6a93-48ba-b9b4-bebb2cefc675-kube-api-access-qjkr2\") pod \"node-exporter-htcj2\" (UID: \"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675\") " pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:03.001200 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.001130    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-htcj2"
Sep 10 19:17:03.013412 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:03.013388    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fdfcb26_6a93_48ba_b9b4_bebb2cefc675.slice/crio-f3ca0e82035b7e6604a5aeb57f6ca84f3d4137286fd239a80aefe426b85fb2f8 WatchSource:0}: Error finding container f3ca0e82035b7e6604a5aeb57f6ca84f3d4137286fd239a80aefe426b85fb2f8: Status 404 returned error can't find the container with id f3ca0e82035b7e6604a5aeb57f6ca84f3d4137286fd239a80aefe426b85fb2f8
Sep 10 19:17:03.125340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.125303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htcj2" event={"ID":"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675","Type":"ContainerStarted","Data":"f3ca0e82035b7e6604a5aeb57f6ca84f3d4137286fd239a80aefe426b85fb2f8"}
Sep 10 19:17:03.312364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.312274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:03.312515 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:03.312443    2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 19:17:03.312555 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:03.312519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls podName:a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.312502819 +0000 UTC m=+83.133558934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-6cpxs" (UID: "a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823") : secret "openshift-state-metrics-tls" not found
Sep 10 19:17:03.736095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.736067    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:03.740943 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.740910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.743433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743380    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 19:17:03.743433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743416    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 19:17:03.743625 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743416    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 19:17:03.743675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743661    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 19:17:03.743730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743702    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zmjht\""
Sep 10 19:17:03.743730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743716    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 19:17:03.743831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743743    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 19:17:03.743831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 19:17:03.743975 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.743958    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 19:17:03.744077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.744060    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 19:17:03.754125 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.754103    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:03.817232 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817197    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817379    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817402    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817516    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.817683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.817621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twr55\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918458    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:03.918518    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle podName:2ac25668-874e-4462-86f2-af405d6c2801 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.41849239 +0000 UTC m=+83.239548530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2ac25668-874e-4462-86f2-af405d6c2801") : configmap references non-existent config key: ca-bundle.crt
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918617    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918922 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twr55\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918922 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.918922 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.918734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.919533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.919506    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.919776 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:03.919757    2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 10 19:17:03.919915 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:03.919903    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls podName:2ac25668-874e-4462-86f2-af405d6c2801 nodeName:}" failed. No retries permitted until 2026-09-10 19:17:04.41988426 +0000 UTC m=+83.240940390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2ac25668-874e-4462-86f2-af405d6c2801") : secret "alertmanager-main-tls" not found
Sep 10 19:17:03.921438 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.921408    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.921438 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.921428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.921599 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.921412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.921717 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.921691    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.921819 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.921703    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.922043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.922026    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.922495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.922477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.923710 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.923690    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:03.929526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:03.929504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twr55\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.090184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.090048    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-psnrk"
Sep 10 19:17:04.323318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.323285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:04.325684 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.325656    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-6cpxs\" (UID: \"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:04.424505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.424461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.424720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.424530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.425344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.425323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.426758 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.426738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.473720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.473685    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"
Sep 10 19:17:04.652140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.652106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:17:04.757668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:04.757632    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs"]
Sep 10 19:17:04.803081 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:04.803050    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ae28ab_8073_4ab4_ab4c_f5a5b3dfd823.slice/crio-b9cf32f1f9ae297facde456bb538bafa1372e659b4bf3199efaf870808bdd7ec WatchSource:0}: Error finding container b9cf32f1f9ae297facde456bb538bafa1372e659b4bf3199efaf870808bdd7ec: Status 404 returned error can't find the container with id b9cf32f1f9ae297facde456bb538bafa1372e659b4bf3199efaf870808bdd7ec
Sep 10 19:17:05.085093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.085065    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:17:05.085562 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:05.085529    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac25668_874e_4462_86f2_af405d6c2801.slice/crio-48bcb28c0beab984ec500b1e1ddcfffefbd8df65800e02aa2f551cba45740cfe WatchSource:0}: Error finding container 48bcb28c0beab984ec500b1e1ddcfffefbd8df65800e02aa2f551cba45740cfe: Status 404 returned error can't find the container with id 48bcb28c0beab984ec500b1e1ddcfffefbd8df65800e02aa2f551cba45740cfe
Sep 10 19:17:05.135514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.135474    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"48bcb28c0beab984ec500b1e1ddcfffefbd8df65800e02aa2f551cba45740cfe"}
Sep 10 19:17:05.137258 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.137229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs" event={"ID":"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823","Type":"ContainerStarted","Data":"52a726d47c3208524108fce9bc216c2a516f8eddf590821565c9f4a015b15a53"}
Sep 10 19:17:05.137397 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.137292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs" event={"ID":"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823","Type":"ContainerStarted","Data":"f1a6ad30e9fc7429a1154105ae42aec7a59a7c24386065775364fff33ea0ad06"}
Sep 10 19:17:05.137397 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.137327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs" event={"ID":"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823","Type":"ContainerStarted","Data":"b9cf32f1f9ae297facde456bb538bafa1372e659b4bf3199efaf870808bdd7ec"}
Sep 10 19:17:05.138839 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.138817    2567 generic.go:356] "Generic (PLEG): container finished" podID="9fdfcb26-6a93-48ba-b9b4-bebb2cefc675" containerID="10dd7f9c61ee0b3aafee571ac18b3bc027b629a0e80795294b78d1a094f6185b" exitCode=0
Sep 10 19:17:05.138913 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.138853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htcj2" event={"ID":"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675","Type":"ContainerDied","Data":"10dd7f9c61ee0b3aafee571ac18b3bc027b629a0e80795294b78d1a094f6185b"}
Sep 10 19:17:05.746281 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.746247    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"]
Sep 10 19:17:05.750497 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.750474    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.753542 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.753513    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 19:17:05.753776 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.753750    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 19:17:05.753897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.753791    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 19:17:05.753897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.753750    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 19:17:05.754677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.754207    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 19:17:05.754677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.754408    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d1llth74ga2jp\""
Sep 10 19:17:05.754677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.754588    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lqftf\""
Sep 10 19:17:05.762107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.762038    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"]
Sep 10 19:17:05.834675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.834849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-metrics-client-ca\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.834849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4gfb\" (UniqueName: \"kubernetes.io/projected/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-kube-api-access-f4gfb\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.834849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834815    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.835047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-grpc-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.835047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.835047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.834904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.835047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.835009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.935627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4gfb\" (UniqueName: \"kubernetes.io/projected/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-kube-api-access-f4gfb\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.935808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.935808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-grpc-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.935808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.935983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.936054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.935992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.936054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.936049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.936166 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.936084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-metrics-client-ca\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.936981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.936874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-metrics-client-ca\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.939253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.939223    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.939371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.939275    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.939371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.939283    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.939371 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.939359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.939514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.939391    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-grpc-tls\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.940685 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.940667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:05.945887 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:05.945865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4gfb\" (UniqueName: \"kubernetes.io/projected/bfdf7929-a3ae-46a5-84ff-0e78602fbb7e-kube-api-access-f4gfb\") pod \"thanos-querier-5d8b5d594c-xtt2c\" (UID: \"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e\") " pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:06.063787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:06.063694    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:06.146156 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:06.146116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htcj2" event={"ID":"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675","Type":"ContainerStarted","Data":"59694c1f1e93d5819cc0f0d7809b65183fb5a3316cfc7d752b1e633180b9c102"}
Sep 10 19:17:06.146340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:06.146176    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-htcj2" event={"ID":"9fdfcb26-6a93-48ba-b9b4-bebb2cefc675","Type":"ContainerStarted","Data":"41ef7c4db1c0d970f860fc0c118e71a9ca0c2054f62d8e2af727c0d39cdb4099"}
Sep 10 19:17:06.167441 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:06.167381    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-htcj2" podStartSLOduration=2.252886383 podStartE2EDuration="4.167362977s" totalImagesPullingTime="1.847805765s" totalInitContainerRuntime="66.670829ms" isStatefulPod=true podCreationTimestamp="2026-09-10 19:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:06.165972105 +0000 UTC m=+84.987028240" watchObservedRunningTime="2026-09-10 19:17:06.167362977 +0000 UTC m=+84.988419110"
Sep 10 19:17:06.634803 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:06.634717    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"]
Sep 10 19:17:06.637676 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:06.637651    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfdf7929_a3ae_46a5_84ff_0e78602fbb7e.slice/crio-9cbb23a11cb2e91c38da92a00d91d7f277eb9f7bc8e0c71e2559d3412baa4f41 WatchSource:0}: Error finding container 9cbb23a11cb2e91c38da92a00d91d7f277eb9f7bc8e0c71e2559d3412baa4f41: Status 404 returned error can't find the container with id 9cbb23a11cb2e91c38da92a00d91d7f277eb9f7bc8e0c71e2559d3412baa4f41
Sep 10 19:17:07.151050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.151005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"9cbb23a11cb2e91c38da92a00d91d7f277eb9f7bc8e0c71e2559d3412baa4f41"}
Sep 10 19:17:07.152568 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.152539    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5" exitCode=0
Sep 10 19:17:07.152703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.152627    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"}
Sep 10 19:17:07.154966 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.154929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs" event={"ID":"a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823","Type":"ContainerStarted","Data":"e9aeed324fc489606a09e03462f51d054263ce741104f0b56b676162c7466f25"}
Sep 10 19:17:07.205241 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.205181    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-6cpxs" podStartSLOduration=3.800016408 podStartE2EDuration="5.205163701s" totalImagesPullingTime="1.405147293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:07.203381226 +0000 UTC m=+86.024437360" watchObservedRunningTime="2026-09-10 19:17:07.205163701 +0000 UTC m=+86.026219852"
Sep 10 19:17:07.442157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.442066    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"]
Sep 10 19:17:07.445792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.445761    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.459195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.459170    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"]
Sep 10 19:17:07.463747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.463724    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 19:17:07.467166 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.467150    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-x9kp8\""
Sep 10 19:17:07.551103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.551063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69cbe945-945b-4a5c-aad3-cc58f899888a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.551276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.551118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttf86\" (UniqueName: \"kubernetes.io/projected/69cbe945-945b-4a5c-aad3-cc58f899888a-kube-api-access-ttf86\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.652538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.652503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69cbe945-945b-4a5c-aad3-cc58f899888a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.652538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.652541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttf86\" (UniqueName: \"kubernetes.io/projected/69cbe945-945b-4a5c-aad3-cc58f899888a-kube-api-access-ttf86\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.655139 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.655112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/69cbe945-945b-4a5c-aad3-cc58f899888a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.666039 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.666005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttf86\" (UniqueName: \"kubernetes.io/projected/69cbe945-945b-4a5c-aad3-cc58f899888a-kube-api-access-ttf86\") pod \"monitoring-plugin-85bbbbbf4-lt52t\" (UID: \"69cbe945-945b-4a5c-aad3-cc58f899888a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:07.756626 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:07.756532    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:08.033498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:08.033467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"]
Sep 10 19:17:08.036638 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:08.036608    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69cbe945_945b_4a5c_aad3_cc58f899888a.slice/crio-7295d6d4e8a07a25a8af474d8325a51e1b7d68fbb1ec2808d20d23de8d687d34 WatchSource:0}: Error finding container 7295d6d4e8a07a25a8af474d8325a51e1b7d68fbb1ec2808d20d23de8d687d34: Status 404 returned error can't find the container with id 7295d6d4e8a07a25a8af474d8325a51e1b7d68fbb1ec2808d20d23de8d687d34
Sep 10 19:17:08.162349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:08.162268    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t" event={"ID":"69cbe945-945b-4a5c-aad3-cc58f899888a","Type":"ContainerStarted","Data":"7295d6d4e8a07a25a8af474d8325a51e1b7d68fbb1ec2808d20d23de8d687d34"}
Sep 10 19:17:08.236980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:08.236926    2567 patch_prober.go:28] interesting pod/image-registry-5b6cf6b8f4-6hbdx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 19:17:08.237172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:08.237009    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:17:09.169465 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.169431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"e4c87fcfa9e1e20e021ca72a12c510c310738fab79d7ed4d6e3e4798c3b49d32"}
Sep 10 19:17:09.172225 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.172203    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"}
Sep 10 19:17:09.984185 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.984154    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:17:09.988756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.988731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:09.992268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992245    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992575    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992607    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992629    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2hqsq\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992732    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992764    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 19:17:09.992603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.992778    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 19:17:09.993334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.993069    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 19:17:09.997869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.997846    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:17:09.998317 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:09.998301    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 19:17:10.060698 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.060668    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:17:10.077112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077307    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077431 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077431 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077396    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfkhg\" (UniqueName: \"kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.077517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.077466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.177979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.177924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178115    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfkhg\" (UniqueName: \"kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.178413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.178301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.179041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.179002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.179157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.179102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.179352 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.179331    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.179657 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.179636    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.180816 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.180786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"8f6e0859f0f55dfa7dfe7a067f77c8596dc80430062a08eaf330aa44dd8692e7"}
Sep 10 19:17:10.180920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.180868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"1410aaccf9a1534e448c3580b887e834ae8388ab5fa3282c877914945f29cb99"}
Sep 10 19:17:10.180920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.180883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.181524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.181501    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.184538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.184514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"}
Sep 10 19:17:10.184628 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.184588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"}
Sep 10 19:17:10.184692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.184630    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"}
Sep 10 19:17:10.184692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.184669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"}
Sep 10 19:17:10.186306 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.186284    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t" event={"ID":"69cbe945-945b-4a5c-aad3-cc58f899888a","Type":"ContainerStarted","Data":"28ce816ab2de191791396792a106ba5ae984cde8a3b3e547520ca40d348df6eb"}
Sep 10 19:17:10.186586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.186545    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:10.188110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.188090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfkhg\" (UniqueName: \"kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg\") pod \"console-6c7c4d4d6d-4kzx9\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") " pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.192435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.192364    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t"
Sep 10 19:17:10.205423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.205383    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-lt52t" podStartSLOduration=1.302583566 podStartE2EDuration="3.205365666s" totalImagesPullingTime="1.9027821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:10.203857733 +0000 UTC m=+89.024913864" watchObservedRunningTime="2026-09-10 19:17:10.205365666 +0000 UTC m=+89.026421799"
Sep 10 19:17:10.299580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.299546    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:10.612788 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:10.612754    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:17:10.615916 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:17:10.615890    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e11372_be56_4a39_94d7_ad7b61b2cace.slice/crio-5e389f67e796cfb6953eff9a9fb3ede4be980567c28edcf87f25ca51fc3d06f3 WatchSource:0}: Error finding container 5e389f67e796cfb6953eff9a9fb3ede4be980567c28edcf87f25ca51fc3d06f3: Status 404 returned error can't find the container with id 5e389f67e796cfb6953eff9a9fb3ede4be980567c28edcf87f25ca51fc3d06f3
Sep 10 19:17:11.191731 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.191693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7c4d4d6d-4kzx9" event={"ID":"49e11372-be56-4a39-94d7-ad7b61b2cace","Type":"ContainerStarted","Data":"5e389f67e796cfb6953eff9a9fb3ede4be980567c28edcf87f25ca51fc3d06f3"}
Sep 10 19:17:11.194715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.194680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"85028397ef978bbcfe62a9a4d88019ebec46904a83ae75d38e2cba5b58afcf17"}
Sep 10 19:17:11.194850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.194751    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"60f2ebf5e4b413bf91820fd2640088859d0a58c486fff2802978c5f9021bfb61"}
Sep 10 19:17:11.194959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.194886    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:11.194959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.194948    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" event={"ID":"bfdf7929-a3ae-46a5-84ff-0e78602fbb7e","Type":"ContainerStarted","Data":"9292522279cad2e7ed5ef91d8fe68779066ddd0bac5719cf82c16456f33f5a43"}
Sep 10 19:17:11.197967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.197925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerStarted","Data":"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"}
Sep 10 19:17:11.222786 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.222740    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c" podStartSLOduration=2.740413434 podStartE2EDuration="6.22272306s" totalImagesPullingTime="3.482309626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:11.220583937 +0000 UTC m=+90.041640069" watchObservedRunningTime="2026-09-10 19:17:11.22272306 +0000 UTC m=+90.043779192"
Sep 10 19:17:11.252909 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:11.252857    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.148989764 podStartE2EDuration="8.252838657s" totalImagesPullingTime="4.088483726s" totalInitContainerRuntime="15.365167ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:11.250871609 +0000 UTC m=+90.071927743" watchObservedRunningTime="2026-09-10 19:17:11.252838657 +0000 UTC m=+90.073894791"
Sep 10 19:17:14.219559 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:14.219524    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7c4d4d6d-4kzx9" event={"ID":"49e11372-be56-4a39-94d7-ad7b61b2cace","Type":"ContainerStarted","Data":"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"}
Sep 10 19:17:14.238406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:14.238354    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c7c4d4d6d-4kzx9" podStartSLOduration=2.231546528 podStartE2EDuration="5.238339549s" totalImagesPullingTime="3.006793021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:17:14.237964419 +0000 UTC m=+93.059020551" watchObservedRunningTime="2026-09-10 19:17:14.238339549 +0000 UTC m=+93.059395682"
Sep 10 19:17:15.296536 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:15.296500    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:17:17.211181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:17.211151    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d8b5d594c-xtt2c"
Sep 10 19:17:20.058268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:20.058234    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4qdbk"
Sep 10 19:17:20.300639 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:20.300559    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:20.300639 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:20.300617    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:20.305730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:20.305702    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:21.253178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:21.253149    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:17:32.294441 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:32.294405    2567 generic.go:356] "Generic (PLEG): container finished" podID="a29dd7c6-d869-4497-9a3a-23a7289a6c58" containerID="9bb5df8ed51f7cfa93f2278c00940f03d69079a76cc3b398054858c553b0c561" exitCode=0
Sep 10 19:17:32.295003 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:32.294473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr" event={"ID":"a29dd7c6-d869-4497-9a3a-23a7289a6c58","Type":"ContainerDied","Data":"9bb5df8ed51f7cfa93f2278c00940f03d69079a76cc3b398054858c553b0c561"}
Sep 10 19:17:32.295003 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:32.294898    2567 scope.go:117] "RemoveContainer" containerID="9bb5df8ed51f7cfa93f2278c00940f03d69079a76cc3b398054858c553b0c561"
Sep 10 19:17:33.300856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:33.300821    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-8rgkr" event={"ID":"a29dd7c6-d869-4497-9a3a-23a7289a6c58","Type":"ContainerStarted","Data":"2e51261b923dc19420c4963fef4e19679e207ad8a64aac908a87be1d135646e7"}
Sep 10 19:17:40.317271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.317206    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerName="registry" containerID="cri-o://68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6" gracePeriod=30
Sep 10 19:17:40.559350 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.559325    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:17:40.649899 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.649863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.649919    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.649977    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650007    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650047    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ddcg\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650083    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650128    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration\") pod \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\" (UID: \"3a29c116-bcd1-4641-b3b1-5ddeadc1e839\") "
Sep 10 19:17:40.650556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650516    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:40.650556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.650527    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:17:40.652709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.652674    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:40.652893 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.652866    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:40.653045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.653017    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:17:40.653102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.653068    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg" (OuterVolumeSpecName: "kube-api-access-8ddcg") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "kube-api-access-8ddcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:40.653140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.653097    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:17:40.659207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.659174    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3a29c116-bcd1-4641-b3b1-5ddeadc1e839" (UID: "3a29c116-bcd1-4641-b3b1-5ddeadc1e839"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:17:40.751551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751507    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ddcg\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-kube-api-access-8ddcg\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751542    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751551    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-bound-sa-token\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751560    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-image-registry-private-configuration\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751571    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-installation-pull-secrets\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751579    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-ca-trust-extracted\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751587    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-registry-certificates\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:40.751821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:40.751595    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3a29c116-bcd1-4641-b3b1-5ddeadc1e839-trusted-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:17:41.337467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.337431    2567 generic.go:356] "Generic (PLEG): container finished" podID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerID="68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6" exitCode=0
Sep 10 19:17:41.337973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.337480    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"
Sep 10 19:17:41.337973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.337508    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" event={"ID":"3a29c116-bcd1-4641-b3b1-5ddeadc1e839","Type":"ContainerDied","Data":"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"}
Sep 10 19:17:41.337973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.337560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx" event={"ID":"3a29c116-bcd1-4641-b3b1-5ddeadc1e839","Type":"ContainerDied","Data":"61583ff6d8af29a693b0b6ef102bae8a8efa690cd63ce32c2a3bfacdb9f99cbd"}
Sep 10 19:17:41.337973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.337579    2567 scope.go:117] "RemoveContainer" containerID="68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"
Sep 10 19:17:41.351857 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.351834    2567 scope.go:117] "RemoveContainer" containerID="68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"
Sep 10 19:17:41.352208 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:17:41.352187    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6\": container with ID starting with 68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6 not found: ID does not exist" containerID="68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"
Sep 10 19:17:41.352309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.352222    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6"} err="failed to get container status \"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6\": rpc error: code = NotFound desc = could not find container \"68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6\": container with ID starting with 68764690d9198cca014b409555831648b145ace2b2af99c55db1a8fbbdb4a0f6 not found: ID does not exist"
Sep 10 19:17:41.363662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.363630    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:17:41.368386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.368357    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b6cf6b8f4-6hbdx"]
Sep 10 19:17:41.701201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:41.701167    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" path="/var/lib/kubelet/pods/3a29c116-bcd1-4641-b3b1-5ddeadc1e839/volumes"
Sep 10 19:17:52.386771 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.386738    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b" containerID="4bb78035e365c794b21126a74bc18bd593bdda9a686af946df5ffc3266a9d7e0" exitCode=0
Sep 10 19:17:52.387242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.386815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb" event={"ID":"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b","Type":"ContainerDied","Data":"4bb78035e365c794b21126a74bc18bd593bdda9a686af946df5ffc3266a9d7e0"}
Sep 10 19:17:52.387242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.387202    2567 scope.go:117] "RemoveContainer" containerID="4bb78035e365c794b21126a74bc18bd593bdda9a686af946df5ffc3266a9d7e0"
Sep 10 19:17:52.388482 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.388456    2567 generic.go:356] "Generic (PLEG): container finished" podID="af80a1bb-0779-48f4-9f78-9802c2015d70" containerID="04e25820b6e25086b19574d49911d6927b486a2e810c904f3a029eeee76905cc" exitCode=0
Sep 10 19:17:52.388598 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.388540    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd" event={"ID":"af80a1bb-0779-48f4-9f78-9802c2015d70","Type":"ContainerDied","Data":"04e25820b6e25086b19574d49911d6927b486a2e810c904f3a029eeee76905cc"}
Sep 10 19:17:52.388917 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:52.388898    2567 scope.go:117] "RemoveContainer" containerID="04e25820b6e25086b19574d49911d6927b486a2e810c904f3a029eeee76905cc"
Sep 10 19:17:53.394620 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:53.394586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dxjsb" event={"ID":"ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b","Type":"ContainerStarted","Data":"a9b9b45001757e2c89008c3ca4a04b989b7627a70de4402e0eddd367684cbc63"}
Sep 10 19:17:53.396488 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:17:53.396467    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5mnjd" event={"ID":"af80a1bb-0779-48f4-9f78-9802c2015d70","Type":"ContainerStarted","Data":"e27bac26c9e3db55ecf047530531cb63f4d8e2e9e494063fff261ddedcc711d1"}
Sep 10 19:18:23.029233 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029197    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029474    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="alertmanager" containerID="cri-o://286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b" gracePeriod=120
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029532    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-metric" containerID="cri-o://7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75" gracePeriod=120
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029564    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy" containerID="cri-o://00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f" gracePeriod=120
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029583    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="config-reloader" containerID="cri-o://dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21" gracePeriod=120
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029540    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-web" containerID="cri-o://95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b" gracePeriod=120
Sep 10 19:18:23.029722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.029592    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="prom-label-proxy" containerID="cri-o://4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3" gracePeriod=120
Sep 10 19:18:23.522376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522341    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3" exitCode=0
Sep 10 19:18:23.522376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522365    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f" exitCode=0
Sep 10 19:18:23.522376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522371    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21" exitCode=0
Sep 10 19:18:23.522376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522376    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b" exitCode=0
Sep 10 19:18:23.522641 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522413    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"}
Sep 10 19:18:23.522641 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522478    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"}
Sep 10 19:18:23.522641 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522506    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"}
Sep 10 19:18:23.522641 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:23.522529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"}
Sep 10 19:18:24.271672 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.271647    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.421365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421333    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421372    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421402    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421522    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421557    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421597    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twr55\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421625    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.421896 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421858    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:24.422015 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.421993    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.422075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.422034    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.422127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.422095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.422178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.422166    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.422231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.422206    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web\") pod \"2ac25668-874e-4462-86f2-af405d6c2801\" (UID: \"2ac25668-874e-4462-86f2-af405d6c2801\") "
Sep 10 19:18:24.422498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.422476    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-metrics-client-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.424401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:24.424820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424594    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume" (OuterVolumeSpecName: "config-volume") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.424820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424595    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55" (OuterVolumeSpecName: "kube-api-access-twr55") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "kube-api-access-twr55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:18:24.424820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424642    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.425073 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424880    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.425073 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.424953    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.425073 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.425035    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:18:24.425073 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.425050    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:18:24.425288 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.425236    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.426826 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.426801    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out" (OuterVolumeSpecName: "config-out") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:18:24.429583 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.429560    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.435268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.435248    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config" (OuterVolumeSpecName: "web-config") pod "2ac25668-874e-4462-86f2-af405d6c2801" (UID: "2ac25668-874e-4462-86f2-af405d6c2801"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:18:24.523103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523072    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523098    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-main-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523107    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523116    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-tls-assets\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523125    2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-config-volume\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523133    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523142    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twr55\" (UniqueName: \"kubernetes.io/projected/2ac25668-874e-4462-86f2-af405d6c2801-kube-api-access-twr55\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523153    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-web-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523163    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523171    2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2ac25668-874e-4462-86f2-af405d6c2801-cluster-tls-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523179    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-config-out\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.523333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.523187    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2ac25668-874e-4462-86f2-af405d6c2801-alertmanager-main-db\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:18:24.529504 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529474    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75" exitCode=0
Sep 10 19:18:24.529504 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529503    2567 generic.go:356] "Generic (PLEG): container finished" podID="2ac25668-874e-4462-86f2-af405d6c2801" containerID="95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b" exitCode=0
Sep 10 19:18:24.529702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529551    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.529702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"}
Sep 10 19:18:24.529702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529629    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"}
Sep 10 19:18:24.529702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529656    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2ac25668-874e-4462-86f2-af405d6c2801","Type":"ContainerDied","Data":"48bcb28c0beab984ec500b1e1ddcfffefbd8df65800e02aa2f551cba45740cfe"}
Sep 10 19:18:24.529702 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.529688    2567 scope.go:117] "RemoveContainer" containerID="4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"
Sep 10 19:18:24.537706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.537682    2567 scope.go:117] "RemoveContainer" containerID="7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"
Sep 10 19:18:24.545295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.545274    2567 scope.go:117] "RemoveContainer" containerID="00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"
Sep 10 19:18:24.552336 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.552316    2567 scope.go:117] "RemoveContainer" containerID="95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"
Sep 10 19:18:24.557080 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.557054    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:24.561892 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.561789    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:24.562062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.561912    2567 scope.go:117] "RemoveContainer" containerID="dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"
Sep 10 19:18:24.569509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.569491    2567 scope.go:117] "RemoveContainer" containerID="286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"
Sep 10 19:18:24.576871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.576853    2567 scope.go:117] "RemoveContainer" containerID="115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"
Sep 10 19:18:24.584540 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.584521    2567 scope.go:117] "RemoveContainer" containerID="4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"
Sep 10 19:18:24.584824 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.584804    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3\": container with ID starting with 4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3 not found: ID does not exist" containerID="4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"
Sep 10 19:18:24.584874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.584835    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"} err="failed to get container status \"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3\": rpc error: code = NotFound desc = could not find container \"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3\": container with ID starting with 4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3 not found: ID does not exist"
Sep 10 19:18:24.584874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.584854    2567 scope.go:117] "RemoveContainer" containerID="7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"
Sep 10 19:18:24.585152 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.585137    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75\": container with ID starting with 7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75 not found: ID does not exist" containerID="7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"
Sep 10 19:18:24.585205 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585156    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"} err="failed to get container status \"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75\": rpc error: code = NotFound desc = could not find container \"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75\": container with ID starting with 7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75 not found: ID does not exist"
Sep 10 19:18:24.585205 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585172    2567 scope.go:117] "RemoveContainer" containerID="00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"
Sep 10 19:18:24.585420 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.585403    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f\": container with ID starting with 00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f not found: ID does not exist" containerID="00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"
Sep 10 19:18:24.585487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585428    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"} err="failed to get container status \"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f\": rpc error: code = NotFound desc = could not find container \"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f\": container with ID starting with 00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f not found: ID does not exist"
Sep 10 19:18:24.585487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585452    2567 scope.go:117] "RemoveContainer" containerID="95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"
Sep 10 19:18:24.585723 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.585706    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b\": container with ID starting with 95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b not found: ID does not exist" containerID="95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"
Sep 10 19:18:24.585759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585728    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"} err="failed to get container status \"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b\": rpc error: code = NotFound desc = could not find container \"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b\": container with ID starting with 95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b not found: ID does not exist"
Sep 10 19:18:24.585759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585742    2567 scope.go:117] "RemoveContainer" containerID="dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"
Sep 10 19:18:24.585992 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.585977    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21\": container with ID starting with dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21 not found: ID does not exist" containerID="dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"
Sep 10 19:18:24.586043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.585995    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"} err="failed to get container status \"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21\": rpc error: code = NotFound desc = could not find container \"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21\": container with ID starting with dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21 not found: ID does not exist"
Sep 10 19:18:24.586043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586009    2567 scope.go:117] "RemoveContainer" containerID="286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"
Sep 10 19:18:24.586202 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.586185    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b\": container with ID starting with 286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b not found: ID does not exist" containerID="286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"
Sep 10 19:18:24.586265 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586209    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"} err="failed to get container status \"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b\": rpc error: code = NotFound desc = could not find container \"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b\": container with ID starting with 286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b not found: ID does not exist"
Sep 10 19:18:24.586265 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586230    2567 scope.go:117] "RemoveContainer" containerID="115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"
Sep 10 19:18:24.586488 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:18:24.586472    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5\": container with ID starting with 115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5 not found: ID does not exist" containerID="115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"
Sep 10 19:18:24.586525 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586492    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"} err="failed to get container status \"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5\": rpc error: code = NotFound desc = could not find container \"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5\": container with ID starting with 115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5 not found: ID does not exist"
Sep 10 19:18:24.586525 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586505    2567 scope.go:117] "RemoveContainer" containerID="4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"
Sep 10 19:18:24.586718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586700    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3"} err="failed to get container status \"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3\": rpc error: code = NotFound desc = could not find container \"4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3\": container with ID starting with 4bf7783cbf27ec3ed093a9c6a04f627da11f4a9cde6884ffe533f7479414a5f3 not found: ID does not exist"
Sep 10 19:18:24.586791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586720    2567 scope.go:117] "RemoveContainer" containerID="7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"
Sep 10 19:18:24.586978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586961    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75"} err="failed to get container status \"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75\": rpc error: code = NotFound desc = could not find container \"7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75\": container with ID starting with 7bdf53d9ec3a2bb1e4353ad0b60d8415b9ba453b631bf673af6187f1ea32cc75 not found: ID does not exist"
Sep 10 19:18:24.587032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.586979    2567 scope.go:117] "RemoveContainer" containerID="00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"
Sep 10 19:18:24.587199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.587181    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f"} err="failed to get container status \"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f\": rpc error: code = NotFound desc = could not find container \"00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f\": container with ID starting with 00e74e1832c6e6ac07479a87caa25a0672c70cddc0c9b3ecc7320dadb8d5df0f not found: ID does not exist"
Sep 10 19:18:24.587297 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.587199    2567 scope.go:117] "RemoveContainer" containerID="95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"
Sep 10 19:18:24.587517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.587487    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b"} err="failed to get container status \"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b\": rpc error: code = NotFound desc = could not find container \"95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b\": container with ID starting with 95c37f1544d3674b335aa8e28ee3f47a4a7321000adfc6a42cc5ca579b2bec5b not found: ID does not exist"
Sep 10 19:18:24.587517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.587510    2567 scope.go:117] "RemoveContainer" containerID="dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"
Sep 10 19:18:24.588206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.587972    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21"} err="failed to get container status \"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21\": rpc error: code = NotFound desc = could not find container \"dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21\": container with ID starting with dee6fcb2eb70296b1ef8ba21ccb8b325a0437e020267d9b6123cbfa217e17a21 not found: ID does not exist"
Sep 10 19:18:24.588206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.588005    2567 scope.go:117] "RemoveContainer" containerID="286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"
Sep 10 19:18:24.588366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.588345    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b"} err="failed to get container status \"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b\": rpc error: code = NotFound desc = could not find container \"286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b\": container with ID starting with 286f2ee6c22c460404847a8728504544fb0168b1348abbd3db52fbbf86c2a83b not found: ID does not exist"
Sep 10 19:18:24.588366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.588365    2567 scope.go:117] "RemoveContainer" containerID="115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"
Sep 10 19:18:24.588608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.588582    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5"} err="failed to get container status \"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5\": rpc error: code = NotFound desc = could not find container \"115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5\": container with ID starting with 115d38f44da87ff1a9bc5fdcba3c29cf19ab7db0953023acf9a8c70301bf71b5 not found: ID does not exist"
Sep 10 19:18:24.589736 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.589718    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:24.590093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590079    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="alertmanager"
Sep 10 19:18:24.590093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590094    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="alertmanager"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590103    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590109    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590118    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="config-reloader"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590123    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="config-reloader"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590136    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-web"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590141    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-web"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590150    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerName="registry"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590154    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerName="registry"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590160    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-metric"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590166    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-metric"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590174    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="init-config-reloader"
Sep 10 19:18:24.590172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590179    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="init-config-reloader"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590186    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="prom-label-proxy"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590190    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="prom-label-proxy"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590256    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="alertmanager"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590265    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a29c116-bcd1-4641-b3b1-5ddeadc1e839" containerName="registry"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590271    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="config-reloader"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590276    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-web"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590282    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590288    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="kube-rbac-proxy-metric"
Sep 10 19:18:24.590537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.590294    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac25668-874e-4462-86f2-af405d6c2801" containerName="prom-label-proxy"
Sep 10 19:18:24.598855 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.598825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.601675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601651    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 19:18:24.601675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601666    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 19:18:24.601865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601669    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 19:18:24.601865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601814    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 19:18:24.601865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601817    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 19:18:24.602049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.601995    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 19:18:24.602049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.602022    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zmjht\""
Sep 10 19:18:24.602275 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.602250    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 19:18:24.602329 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.602262    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 19:18:24.607375 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.607353    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:24.607483 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.607436    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 19:18:24.724725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724672    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724858    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xh49\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-kube-api-access-7xh49\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.724978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.725168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.724999    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.725168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.725022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.725168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.725077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.725168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.725095    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.825707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825657    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.825707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825705    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xh49\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-kube-api-access-7xh49\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.825707 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825802    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825893    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.825982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.826015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.826047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.826561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.826362    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.827262 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.826921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.827262 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.826986    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.828851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.828825    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-out\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829001 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.828980    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-web-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829229 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829198    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829202    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829497 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829670 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.829717 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.829702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.830728 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.830711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.834675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.834657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xh49\" (UniqueName: \"kubernetes.io/projected/f8a7a297-4f9d-40fc-a25d-6e9872d302fc-kube-api-access-7xh49\") pod \"alertmanager-main-0\" (UID: \"f8a7a297-4f9d-40fc-a25d-6e9872d302fc\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:24.909876 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:24.909839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 19:18:25.194008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:25.193919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 19:18:25.196386 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:18:25.196357    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a7a297_4f9d_40fc_a25d_6e9872d302fc.slice/crio-90724da4eb4e7323a74d82662d9a4c48b56d6e5d19193549044f81030ce06c13 WatchSource:0}: Error finding container 90724da4eb4e7323a74d82662d9a4c48b56d6e5d19193549044f81030ce06c13: Status 404 returned error can't find the container with id 90724da4eb4e7323a74d82662d9a4c48b56d6e5d19193549044f81030ce06c13
Sep 10 19:18:25.536510 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:25.536470    2567 generic.go:356] "Generic (PLEG): container finished" podID="f8a7a297-4f9d-40fc-a25d-6e9872d302fc" containerID="4d49370d6690c57b6cd32ffe79d3a8f3ef65958892dbd6616a385408e5f27510" exitCode=0
Sep 10 19:18:25.536888 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:25.536533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerDied","Data":"4d49370d6690c57b6cd32ffe79d3a8f3ef65958892dbd6616a385408e5f27510"}
Sep 10 19:18:25.536888 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:25.536585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"90724da4eb4e7323a74d82662d9a4c48b56d6e5d19193549044f81030ce06c13"}
Sep 10 19:18:25.701297 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:25.701265    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac25668-874e-4462-86f2-af405d6c2801" path="/var/lib/kubelet/pods/2ac25668-874e-4462-86f2-af405d6c2801/volumes"
Sep 10 19:18:26.544519 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"37711d016f1b819bff7b214e7a4e2b797b1ffe0deec1353e28afbc1b84ec9e1b"}
Sep 10 19:18:26.545040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"ffeed7d4790957685bdab541596f33e0efbc380777c2eb7ba6ddd43c3ee1cfcf"}
Sep 10 19:18:26.545040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"f996fde263c0e099036f768bc1a7d074bd0d4e18c7b93303ed479e01feb12d66"}
Sep 10 19:18:26.545040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"bf74935675da5ee83cd23f1361e34e1cf0225ae27738b0bef650190563ae9a5b"}
Sep 10 19:18:26.545040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544685    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"dea27129f2047a4be4d01785c3d212cc459fb483f9a175093845d34471c2bfed"}
Sep 10 19:18:26.545040 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.544726    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f8a7a297-4f9d-40fc-a25d-6e9872d302fc","Type":"ContainerStarted","Data":"636b3c5cd22e40e1002b16a5258e1e5cf73509ef826ffc2f58e51322eaa1431f"}
Sep 10 19:18:26.576972 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:26.574686    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.5578711419999998 podStartE2EDuration="2.574663487s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.792345ms" isStatefulPod=false podCreationTimestamp="2026-09-10 19:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:18:26.571901133 +0000 UTC m=+165.392957277" watchObservedRunningTime="2026-09-10 19:18:26.574663487 +0000 UTC m=+165.395719621"
Sep 10 19:18:27.051924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.051887    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"]
Sep 10 19:18:27.055644 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.055621    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.058491 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.058435    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kswp6\""
Sep 10 19:18:27.058491 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.058484    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 19:18:27.058719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.058524    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 19:18:27.058719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.058542    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 19:18:27.058719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.058484    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 19:18:27.059113 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.059092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 19:18:27.064412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.064389    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 19:18:27.068064 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.068042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"]
Sep 10 19:18:27.147264 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147224    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147274    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-serving-certs-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147302    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-metrics-client-ca\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6sr\" (UniqueName: \"kubernetes.io/projected/6d1969e6-1486-43b5-84eb-a1364012647b-kube-api-access-xl6sr\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-federate-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.147569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.147485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-federate-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-serving-certs-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-metrics-client-ca\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.248725 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.248633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6sr\" (UniqueName: \"kubernetes.io/projected/6d1969e6-1486-43b5-84eb-a1364012647b-kube-api-access-xl6sr\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.249456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.249424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-serving-certs-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.249592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.249562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.249684 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.249666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d1969e6-1486-43b5-84eb-a1364012647b-metrics-client-ca\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.251283 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.251254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.251283 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.251271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-federate-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.251502 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.251483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-secret-telemeter-client\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.251544 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.251499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6d1969e6-1486-43b5-84eb-a1364012647b-telemeter-client-tls\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.257763 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.257738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6sr\" (UniqueName: \"kubernetes.io/projected/6d1969e6-1486-43b5-84eb-a1364012647b-kube-api-access-xl6sr\") pod \"telemeter-client-75d6f8c69b-f9psk\" (UID: \"6d1969e6-1486-43b5-84eb-a1364012647b\") " pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.367382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.367308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"
Sep 10 19:18:27.645521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:27.645491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-75d6f8c69b-f9psk"]
Sep 10 19:18:27.649520 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:18:27.649491    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1969e6_1486_43b5_84eb_a1364012647b.slice/crio-0ba8786c11fdf056138fdd7837600b14c02dd3e1150c8d909317eb3a25793749 WatchSource:0}: Error finding container 0ba8786c11fdf056138fdd7837600b14c02dd3e1150c8d909317eb3a25793749: Status 404 returned error can't find the container with id 0ba8786c11fdf056138fdd7837600b14c02dd3e1150c8d909317eb3a25793749
Sep 10 19:18:28.556274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:28.556229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk" event={"ID":"6d1969e6-1486-43b5-84eb-a1364012647b","Type":"ContainerStarted","Data":"0ba8786c11fdf056138fdd7837600b14c02dd3e1150c8d909317eb3a25793749"}
Sep 10 19:18:29.563587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:29.563556    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk" event={"ID":"6d1969e6-1486-43b5-84eb-a1364012647b","Type":"ContainerStarted","Data":"43e180a68d3e30742af3e3826ecb7b2be7f9e47784a0def45ee53800af40929e"}
Sep 10 19:18:29.563951 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:29.563609    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk" event={"ID":"6d1969e6-1486-43b5-84eb-a1364012647b","Type":"ContainerStarted","Data":"d5dc1b72c094a31971c87e70c23cb9a057e4ca6a4dfb61b01f81613f74f7a533"}
Sep 10 19:18:30.570159 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.570124    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk" event={"ID":"6d1969e6-1486-43b5-84eb-a1364012647b","Type":"ContainerStarted","Data":"b4d3b2bccded49027409da8467204ca6a6a20f0ada67beb5c183c17a456e10b9"}
Sep 10 19:18:30.592841 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.592790    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-75d6f8c69b-f9psk" podStartSLOduration=1.869270663 podStartE2EDuration="3.592773478s" totalImagesPullingTime="1.723502815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:18:30.591423186 +0000 UTC m=+169.412479310" watchObservedRunningTime="2026-09-10 19:18:30.592773478 +0000 UTC m=+169.413829612"
Sep 10 19:18:30.685620 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.685590    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:18:30.689273 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.689249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.699830 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.699801    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:18:30.782244 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782560 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782560 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782464    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.782560 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.782546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjp2p\" (UniqueName: \"kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.882997    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883037    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjp2p\" (UniqueName: \"kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883362 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883891    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.883923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883915    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.884115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.883899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.884115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.884019    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.885680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.885657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.885846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.885830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:30.891430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.891407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjp2p\" (UniqueName: \"kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p\") pod \"console-f6ccb8767-b8fqp\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") " pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:31.000041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:30.999999    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:31.281487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:31.281462    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:18:31.283832 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:18:31.283805    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd6e0c6_aa69_4f0d_a976_3abb2abc3c13.slice/crio-41dc93e426e0753e598a3053e042a3f2bfd6759011b01f3bdf5c23d82dc4bbaa WatchSource:0}: Error finding container 41dc93e426e0753e598a3053e042a3f2bfd6759011b01f3bdf5c23d82dc4bbaa: Status 404 returned error can't find the container with id 41dc93e426e0753e598a3053e042a3f2bfd6759011b01f3bdf5c23d82dc4bbaa
Sep 10 19:18:31.576805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:31.576706    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6ccb8767-b8fqp" event={"ID":"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13","Type":"ContainerStarted","Data":"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"}
Sep 10 19:18:31.576805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:31.576757    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6ccb8767-b8fqp" event={"ID":"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13","Type":"ContainerStarted","Data":"41dc93e426e0753e598a3053e042a3f2bfd6759011b01f3bdf5c23d82dc4bbaa"}
Sep 10 19:18:31.593993 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:31.593920    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f6ccb8767-b8fqp" podStartSLOduration=1.593901905 podStartE2EDuration="1.593901905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:18:31.593809505 +0000 UTC m=+170.414865643" watchObservedRunningTime="2026-09-10 19:18:31.593901905 +0000 UTC m=+170.414958037"
Sep 10 19:18:41.000954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:41.000840    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:41.000954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:41.000908    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:41.005723 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:41.005695    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:41.628627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:41.628602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:18:41.678681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:18:41.678649    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:19:06.706426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:06.706371    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c7c4d4d6d-4kzx9" podUID="49e11372-be56-4a39-94d7-ad7b61b2cace" containerName="console" containerID="cri-o://2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a" gracePeriod=15
Sep 10 19:19:06.951198 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:06.951173    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:19:07.079998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.079909    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.079998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.079974    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080009    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080042    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfkhg\" (UniqueName: \"kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080068    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080225    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config\") pod \"49e11372-be56-4a39-94d7-ad7b61b2cace\" (UID: \"49e11372-be56-4a39-94d7-ad7b61b2cace\") "
Sep 10 19:19:07.080504 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:07.080649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080517    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-oauth-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.080649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080530    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:07.080649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca" (OuterVolumeSpecName: "service-ca") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:07.080810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.080665    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config" (OuterVolumeSpecName: "console-config") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:19:07.082478 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.082445    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:19:07.082578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.082474    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:19:07.082578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.082479    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg" (OuterVolumeSpecName: "kube-api-access-sfkhg") pod "49e11372-be56-4a39-94d7-ad7b61b2cace" (UID: "49e11372-be56-4a39-94d7-ad7b61b2cace"). InnerVolumeSpecName "kube-api-access-sfkhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:19:07.181587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181534    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfkhg\" (UniqueName: \"kubernetes.io/projected/49e11372-be56-4a39-94d7-ad7b61b2cace-kube-api-access-sfkhg\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.181587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181582    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.181587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181593    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.181587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181603    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/49e11372-be56-4a39-94d7-ad7b61b2cace-console-oauth-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.181862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181614    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-console-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.181862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.181623    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e11372-be56-4a39-94d7-ad7b61b2cace-trusted-ca-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:19:07.748073 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.748040    2567 generic.go:356] "Generic (PLEG): container finished" podID="49e11372-be56-4a39-94d7-ad7b61b2cace" containerID="2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a" exitCode=0
Sep 10 19:19:07.748463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.748088    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7c4d4d6d-4kzx9"
Sep 10 19:19:07.748463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.748125    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7c4d4d6d-4kzx9" event={"ID":"49e11372-be56-4a39-94d7-ad7b61b2cace","Type":"ContainerDied","Data":"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"}
Sep 10 19:19:07.748463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.748175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7c4d4d6d-4kzx9" event={"ID":"49e11372-be56-4a39-94d7-ad7b61b2cace","Type":"ContainerDied","Data":"5e389f67e796cfb6953eff9a9fb3ede4be980567c28edcf87f25ca51fc3d06f3"}
Sep 10 19:19:07.748463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.748195    2567 scope.go:117] "RemoveContainer" containerID="2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"
Sep 10 19:19:07.756680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.756660    2567 scope.go:117] "RemoveContainer" containerID="2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"
Sep 10 19:19:07.756976 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:19:07.756959    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a\": container with ID starting with 2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a not found: ID does not exist" containerID="2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"
Sep 10 19:19:07.757017 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.756988    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a"} err="failed to get container status \"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a\": rpc error: code = NotFound desc = could not find container \"2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a\": container with ID starting with 2c51d7e00988617eb8a990cde29a33d5626984caa1b475707d586ac1bdd2d75a not found: ID does not exist"
Sep 10 19:19:07.765658 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.765637    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:19:07.770002 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:07.769976    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c7c4d4d6d-4kzx9"]
Sep 10 19:19:09.700790 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:09.700759    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e11372-be56-4a39-94d7-ad7b61b2cace" path="/var/lib/kubelet/pods/49e11372-be56-4a39-94d7-ad7b61b2cace/volumes"
Sep 10 19:19:51.480026 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.479989    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:19:51.480622 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.480565    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e11372-be56-4a39-94d7-ad7b61b2cace" containerName="console"
Sep 10 19:19:51.480622 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.480588    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e11372-be56-4a39-94d7-ad7b61b2cace" containerName="console"
Sep 10 19:19:51.480756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.480700    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e11372-be56-4a39-94d7-ad7b61b2cace" containerName="console"
Sep 10 19:19:51.485256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.485231    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.491690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.491665    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:19:51.561645 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561645 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561648    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clw6p\" (UniqueName: \"kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561672    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561786    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.561886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.561801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662443 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662443 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clw6p\" (UniqueName: \"kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.662709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.662592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.663369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.663340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.663495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.663340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.663495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.663348    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.663495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.663423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.665057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.665031    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.665142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.665078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.672533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.672513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clw6p\" (UniqueName: \"kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p\") pod \"console-56fc889fdb-2bzsf\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") " pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:51.797123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:51.797031    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:19:52.073004 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:52.072911    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:19:52.077004 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:19:52.076976    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995f76d5_68f1_4be3_9cf6_0d02b5b1c3e7.slice/crio-834b620c661b36b74cf8793ac1321008318355d9ac411d520d384d23cb53624a WatchSource:0}: Error finding container 834b620c661b36b74cf8793ac1321008318355d9ac411d520d384d23cb53624a: Status 404 returned error can't find the container with id 834b620c661b36b74cf8793ac1321008318355d9ac411d520d384d23cb53624a
Sep 10 19:19:52.953272 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:52.953239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc889fdb-2bzsf" event={"ID":"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7","Type":"ContainerStarted","Data":"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"}
Sep 10 19:19:52.953650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:52.953290    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc889fdb-2bzsf" event={"ID":"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7","Type":"ContainerStarted","Data":"834b620c661b36b74cf8793ac1321008318355d9ac411d520d384d23cb53624a"}
Sep 10 19:19:52.973045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:19:52.972992    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56fc889fdb-2bzsf" podStartSLOduration=1.9729766579999999 podStartE2EDuration="1.972976658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:19:52.971695694 +0000 UTC m=+251.792751831" watchObservedRunningTime="2026-09-10 19:19:52.972976658 +0000 UTC m=+251.794032791"
Sep 10 19:20:01.797554 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:01.797515    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:20:01.798041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:01.797645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:20:01.802570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:01.802546    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:20:01.997564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:01.997535    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:20:02.048349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:02.048266    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:20:27.074039 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.073997    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f6ccb8767-b8fqp" podUID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" containerName="console" containerID="cri-o://7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f" gracePeriod=15
Sep 10 19:20:27.315115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.315087    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:20:27.487862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.487821    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.487862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.487863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488147 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.487887    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjp2p\" (UniqueName: \"kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488147 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.487910    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488147 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488133    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488228    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488263    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config\") pod \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\" (UID: \"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13\") "
Sep 10 19:20:27.488319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488296    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:20:27.488474 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488310    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config" (OuterVolumeSpecName: "console-config") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:20:27.488474 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488337    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca" (OuterVolumeSpecName: "service-ca") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:20:27.488599 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488576    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-oauth-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.488649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488604    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.488649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488619    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.488649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.488579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:20:27.490199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.490174    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p" (OuterVolumeSpecName: "kube-api-access-rjp2p") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "kube-api-access-rjp2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:20:27.490314 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.490298    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:20:27.490373 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.490359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" (UID: "5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:20:27.596256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.596221    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.596256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.596253    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-console-oauth-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.596256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.596264    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjp2p\" (UniqueName: \"kubernetes.io/projected/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-kube-api-access-rjp2p\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:27.596500 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:27.596275    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13-trusted-ca-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:28.107456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.107420    2567 generic.go:356] "Generic (PLEG): container finished" podID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" containerID="7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f" exitCode=0
Sep 10 19:20:28.107895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.107480    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f6ccb8767-b8fqp"
Sep 10 19:20:28.107895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.107487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6ccb8767-b8fqp" event={"ID":"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13","Type":"ContainerDied","Data":"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"}
Sep 10 19:20:28.107895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.107528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f6ccb8767-b8fqp" event={"ID":"5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13","Type":"ContainerDied","Data":"41dc93e426e0753e598a3053e042a3f2bfd6759011b01f3bdf5c23d82dc4bbaa"}
Sep 10 19:20:28.107895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.107548    2567 scope.go:117] "RemoveContainer" containerID="7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"
Sep 10 19:20:28.115815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.115795    2567 scope.go:117] "RemoveContainer" containerID="7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"
Sep 10 19:20:28.116143 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:20:28.116122    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f\": container with ID starting with 7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f not found: ID does not exist" containerID="7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"
Sep 10 19:20:28.116227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.116150    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f"} err="failed to get container status \"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f\": rpc error: code = NotFound desc = could not find container \"7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f\": container with ID starting with 7812d98a111d67ba142960e6d4e550ad09898b1937789fe1f4f7f2334d60476f not found: ID does not exist"
Sep 10 19:20:28.125251 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.125227    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:20:28.129518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:28.129497    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f6ccb8767-b8fqp"]
Sep 10 19:20:29.701035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:29.701003    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" path="/var/lib/kubelet/pods/5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13/volumes"
Sep 10 19:20:37.476221 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.476184    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"]
Sep 10 19:20:37.476611 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.476572    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" containerName="console"
Sep 10 19:20:37.476611 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.476584    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" containerName="console"
Sep 10 19:20:37.476683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.476657    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dd6e0c6-aa69-4f0d-a976-3abb2abc3c13" containerName="console"
Sep 10 19:20:37.479800 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.479784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.481983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.481960    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:20:37.482096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.482025    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5snlz\""
Sep 10 19:20:37.482772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.482749    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:20:37.487143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.487117    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"]
Sep 10 19:20:37.575160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.575120    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.575160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.575164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.575369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.575247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7wsw\" (UniqueName: \"kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.676632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.676598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7wsw\" (UniqueName: \"kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.676770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.676665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.676770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.676687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.677051 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.677030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.677140 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.677085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.685738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.685715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7wsw\" (UniqueName: \"kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:37.789832 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:37.789743    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:38.062648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:38.062569    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"]
Sep 10 19:20:38.065640 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:20:38.065616    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8900b65_6b79_4b8a_b240_74efb3005381.slice/crio-5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275 WatchSource:0}: Error finding container 5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275: Status 404 returned error can't find the container with id 5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275
Sep 10 19:20:38.156424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:38.156393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerStarted","Data":"5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275"}
Sep 10 19:20:41.608191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:41.608160    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:20:41.611582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:41.611556    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:20:41.616694 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:41.616663    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:20:41.619637 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:41.619619    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:20:41.620498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:41.620473    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 19:20:45.193600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:45.193565    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8900b65-6b79-4b8a-b240-74efb3005381" containerID="ab843474a0225e45381476cfc65c392824147a01c2560d7f664bd2a83bd6869e" exitCode=0
Sep 10 19:20:45.194013 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:45.193624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerDied","Data":"ab843474a0225e45381476cfc65c392824147a01c2560d7f664bd2a83bd6869e"}
Sep 10 19:20:45.194662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:45.194644    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:20:48.210434 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:48.210398    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8900b65-6b79-4b8a-b240-74efb3005381" containerID="3ba090d3010ad0c2347c8913604b69ca491871079c723e22d87163c14bcd5e87" exitCode=0
Sep 10 19:20:48.210434 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:48.210438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerDied","Data":"3ba090d3010ad0c2347c8913604b69ca491871079c723e22d87163c14bcd5e87"}
Sep 10 19:20:55.246356 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:55.246320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerStarted","Data":"456858b3dae850ec56aa164e75272b2ed6b18c14e80e48f77d0601bb8bbe695a"}
Sep 10 19:20:55.266865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:55.266797    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" podStartSLOduration=2.388393007 podStartE2EDuration="18.266774722s" totalImagesPullingTime="15.878381715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:20:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:20:55.265430258 +0000 UTC m=+314.086486394" watchObservedRunningTime="2026-09-10 19:20:55.266774722 +0000 UTC m=+314.087830856"
Sep 10 19:20:56.253574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:56.253538    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8900b65-6b79-4b8a-b240-74efb3005381" containerID="456858b3dae850ec56aa164e75272b2ed6b18c14e80e48f77d0601bb8bbe695a" exitCode=0
Sep 10 19:20:56.253973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:56.253584    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerDied","Data":"456858b3dae850ec56aa164e75272b2ed6b18c14e80e48f77d0601bb8bbe695a"}
Sep 10 19:20:57.389585 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.389562    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:57.452370 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.452338    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7wsw\" (UniqueName: \"kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw\") pod \"b8900b65-6b79-4b8a-b240-74efb3005381\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") "
Sep 10 19:20:57.452548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.452438    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle\") pod \"b8900b65-6b79-4b8a-b240-74efb3005381\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") "
Sep 10 19:20:57.452548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.452473    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util\") pod \"b8900b65-6b79-4b8a-b240-74efb3005381\" (UID: \"b8900b65-6b79-4b8a-b240-74efb3005381\") "
Sep 10 19:20:57.453142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.453117    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle" (OuterVolumeSpecName: "bundle") pod "b8900b65-6b79-4b8a-b240-74efb3005381" (UID: "b8900b65-6b79-4b8a-b240-74efb3005381"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:57.454591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.454563    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw" (OuterVolumeSpecName: "kube-api-access-j7wsw") pod "b8900b65-6b79-4b8a-b240-74efb3005381" (UID: "b8900b65-6b79-4b8a-b240-74efb3005381"). InnerVolumeSpecName "kube-api-access-j7wsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:20:57.456985 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.456958    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util" (OuterVolumeSpecName: "util") pod "b8900b65-6b79-4b8a-b240-74efb3005381" (UID: "b8900b65-6b79-4b8a-b240-74efb3005381"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:20:57.553391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.553292    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:57.553391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.553328    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8900b65-6b79-4b8a-b240-74efb3005381-util\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:57.553391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:57.553340    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7wsw\" (UniqueName: \"kubernetes.io/projected/b8900b65-6b79-4b8a-b240-74efb3005381-kube-api-access-j7wsw\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:20:58.263873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:58.263848    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746"
Sep 10 19:20:58.264141 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:58.263846    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754079746" event={"ID":"b8900b65-6b79-4b8a-b240-74efb3005381","Type":"ContainerDied","Data":"5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275"}
Sep 10 19:20:58.264141 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:20:58.263981    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5194af6d533ac1ece857f6a7b90509995e4cd0cc3399e3ea1cc11bc1f0efe275"
Sep 10 19:21:05.214683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.214640    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"]
Sep 10 19:21:05.215200 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215180    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="util"
Sep 10 19:21:05.215310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215204    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="util"
Sep 10 19:21:05.215310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215221    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="pull"
Sep 10 19:21:05.215310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215229    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="pull"
Sep 10 19:21:05.215310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215258    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="extract"
Sep 10 19:21:05.215310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215266    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="extract"
Sep 10 19:21:05.215548 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.215357    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8900b65-6b79-4b8a-b240-74efb3005381" containerName="extract"
Sep 10 19:21:05.218377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.218357    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.220789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.220764    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:21:05.220905 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.220772    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 19:21:05.220905 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.220772    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tgd9z\""
Sep 10 19:21:05.239283 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.239253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"]
Sep 10 19:21:05.321398 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.321354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d703996b-8629-43ed-b1b9-8cce2d440324-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.321566 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.321479    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkmsx\" (UniqueName: \"kubernetes.io/projected/d703996b-8629-43ed-b1b9-8cce2d440324-kube-api-access-nkmsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.422219 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.422183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d703996b-8629-43ed-b1b9-8cce2d440324-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.422391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.422365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkmsx\" (UniqueName: \"kubernetes.io/projected/d703996b-8629-43ed-b1b9-8cce2d440324-kube-api-access-nkmsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.422642 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.422624    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d703996b-8629-43ed-b1b9-8cce2d440324-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.433308 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.433275    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkmsx\" (UniqueName: \"kubernetes.io/projected/d703996b-8629-43ed-b1b9-8cce2d440324-kube-api-access-nkmsx\") pod \"cert-manager-operator-controller-manager-55c444bf86-l4kh7\" (UID: \"d703996b-8629-43ed-b1b9-8cce2d440324\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.527954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.527856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"
Sep 10 19:21:05.809751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:05.809726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7"]
Sep 10 19:21:05.811715 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:21:05.811688    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd703996b_8629_43ed_b1b9_8cce2d440324.slice/crio-27d1a7d80d6046975196032ee62c990952e39ef515fa09f8df8dc0ab879f03d3 WatchSource:0}: Error finding container 27d1a7d80d6046975196032ee62c990952e39ef515fa09f8df8dc0ab879f03d3: Status 404 returned error can't find the container with id 27d1a7d80d6046975196032ee62c990952e39ef515fa09f8df8dc0ab879f03d3
Sep 10 19:21:06.301351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:06.301316    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7" event={"ID":"d703996b-8629-43ed-b1b9-8cce2d440324","Type":"ContainerStarted","Data":"27d1a7d80d6046975196032ee62c990952e39ef515fa09f8df8dc0ab879f03d3"}
Sep 10 19:21:08.312507 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:08.312466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7" event={"ID":"d703996b-8629-43ed-b1b9-8cce2d440324","Type":"ContainerStarted","Data":"09616ead4b53a5fed20deb7006910ad9363bdd59d689fcbace56b46e9a6353a4"}
Sep 10 19:21:08.335716 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:08.335665    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-l4kh7" podStartSLOduration=1.789370889 podStartE2EDuration="3.335646925s" totalImagesPullingTime="1.546276036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:08.333208946 +0000 UTC m=+327.154265090" watchObservedRunningTime="2026-09-10 19:21:08.335646925 +0000 UTC m=+327.156703057"
Sep 10 19:21:09.267526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.267492    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rt9t2"]
Sep 10 19:21:09.270941 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.270914    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.273315 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.273298    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 19:21:09.273398 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.273298    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 19:21:09.273485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.273466    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hh8qk\""
Sep 10 19:21:09.282986 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.282959    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rt9t2"]
Sep 10 19:21:09.459396 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.459362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c574\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-kube-api-access-6c574\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.459793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.459419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.560637 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.560553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c574\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-kube-api-access-6c574\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.560637 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.560619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.570537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.570503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.570537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.570513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c574\" (UniqueName: \"kubernetes.io/projected/bbfa42ba-de4d-458c-89fb-6e31f8e8a428-kube-api-access-6c574\") pod \"cert-manager-webhook-6486ffd997-rt9t2\" (UID: \"bbfa42ba-de4d-458c-89fb-6e31f8e8a428\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.597527 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.597497    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:09.867917 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:09.867838    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rt9t2"]
Sep 10 19:21:09.871139 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:21:09.871113    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbfa42ba_de4d_458c_89fb_6e31f8e8a428.slice/crio-312262273b68303a4d9cee9ac567a2bcf332a898a0c9137e831bdd41643c4d9e WatchSource:0}: Error finding container 312262273b68303a4d9cee9ac567a2bcf332a898a0c9137e831bdd41643c4d9e: Status 404 returned error can't find the container with id 312262273b68303a4d9cee9ac567a2bcf332a898a0c9137e831bdd41643c4d9e
Sep 10 19:21:10.323794 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:10.323760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2" event={"ID":"bbfa42ba-de4d-458c-89fb-6e31f8e8a428","Type":"ContainerStarted","Data":"312262273b68303a4d9cee9ac567a2bcf332a898a0c9137e831bdd41643c4d9e"}
Sep 10 19:21:13.341378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:13.341336    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2" event={"ID":"bbfa42ba-de4d-458c-89fb-6e31f8e8a428","Type":"ContainerStarted","Data":"86ea71476c77bd3ec925fedba32b62254dc3b8ff8138d83abea3469012137236"}
Sep 10 19:21:13.341843 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:13.341406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:13.359014 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:13.358961    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2" podStartSLOduration=1.9203727860000002 podStartE2EDuration="4.358945704s" totalImagesPullingTime="2.438572918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:21:13.357506301 +0000 UTC m=+332.178562434" watchObservedRunningTime="2026-09-10 19:21:13.358945704 +0000 UTC m=+332.180001828"
Sep 10 19:21:19.348541 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:19.348508    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rt9t2"
Sep 10 19:21:39.600353 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.600261    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"]
Sep 10 19:21:39.603881 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.603855    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.606112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.606090    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:21:39.606762 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.606744    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:21:39.606852 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.606774    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5snlz\""
Sep 10 19:21:39.611979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.611959    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"]
Sep 10 19:21:39.718131 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.718088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9fjm\" (UniqueName: \"kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.718131 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.718129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.718337 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.718249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.819497 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.819460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9fjm\" (UniqueName: \"kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.819497 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.819496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.819737 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.819539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.819927 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.819908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.820037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.819966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.828967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.828923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9fjm\" (UniqueName: \"kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:39.913799 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:39.913765    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:40.185762 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:40.185739    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"]
Sep 10 19:21:40.188161 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:21:40.188129    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e1b9d4d_0479_4ad6_a4c1_d05b4ce528ac.slice/crio-7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7 WatchSource:0}: Error finding container 7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7: Status 404 returned error can't find the container with id 7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7
Sep 10 19:21:40.474107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:40.474024    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerID="47f687e81dd5108bda06a0b1bd1909a23147a8a9a928bb098989228a7d37d6cf" exitCode=0
Sep 10 19:21:40.474242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:40.474116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d" event={"ID":"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac","Type":"ContainerDied","Data":"47f687e81dd5108bda06a0b1bd1909a23147a8a9a928bb098989228a7d37d6cf"}
Sep 10 19:21:40.474242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:40.474171    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d" event={"ID":"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac","Type":"ContainerStarted","Data":"7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7"}
Sep 10 19:21:41.480882 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:41.480845    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerID="bf21e66bff308985e22aebe0b943992b6db8ffe4a84e3c5ce3160668d931e157" exitCode=0
Sep 10 19:21:41.481305 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:41.480947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d" event={"ID":"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac","Type":"ContainerDied","Data":"bf21e66bff308985e22aebe0b943992b6db8ffe4a84e3c5ce3160668d931e157"}
Sep 10 19:21:42.487273 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:42.487230    2567 generic.go:356] "Generic (PLEG): container finished" podID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerID="48c62aa509fd30a5696ef94da53800323e53c72d72ffff1a1cd70b6ff1ae39e1" exitCode=0
Sep 10 19:21:42.487646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:42.487317    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d" event={"ID":"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac","Type":"ContainerDied","Data":"48c62aa509fd30a5696ef94da53800323e53c72d72ffff1a1cd70b6ff1ae39e1"}
Sep 10 19:21:43.629398 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.629373    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:43.755259 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.755174    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util\") pod \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") "
Sep 10 19:21:43.755259 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.755220    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9fjm\" (UniqueName: \"kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm\") pod \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") "
Sep 10 19:21:43.755490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.755263    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle\") pod \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\" (UID: \"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac\") "
Sep 10 19:21:43.756204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.756175    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle" (OuterVolumeSpecName: "bundle") pod "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" (UID: "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:43.757443 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.757422    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm" (OuterVolumeSpecName: "kube-api-access-p9fjm") pod "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" (UID: "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac"). InnerVolumeSpecName "kube-api-access-p9fjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:43.761267 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.761242    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util" (OuterVolumeSpecName: "util") pod "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" (UID: "2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:43.856378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.856339    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:43.856378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.856370    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-util\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:43.856378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:43.856379    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9fjm\" (UniqueName: \"kubernetes.io/projected/2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac-kube-api-access-p9fjm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:44.498292 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:44.498259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d" event={"ID":"2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac","Type":"ContainerDied","Data":"7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7"}
Sep 10 19:21:44.498476 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:44.498305    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg9f9d"
Sep 10 19:21:44.498476 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:44.498311    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b5816f0976768501d5562e7b07ad19c4d0c2642cf267f4942181755ff85e7b7"
Sep 10 19:21:53.816637 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.816603    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"]
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.816968    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="extract"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.816981    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="extract"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.816995    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="pull"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.817000    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="pull"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.817014    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="util"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.817019    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="util"
Sep 10 19:21:53.817143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.817080    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e1b9d4d-0479-4ad6-a4c1-d05b4ce528ac" containerName="extract"
Sep 10 19:21:53.821535 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.821506    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:53.824365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.824341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:21:53.824503 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.824342    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5snlz\""
Sep 10 19:21:53.825163 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.825115    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:21:53.887883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.887853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"]
Sep 10 19:21:53.945469 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.945433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:53.945649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.945475    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:53.945649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:53.945503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrtj\" (UniqueName: \"kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.042793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.042759    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"]
Sep 10 19:21:54.046212 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046193    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.046271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrtj\" (UniqueName: \"kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.046324 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.046373 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046336    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.046851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046817    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.046851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.046841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.049155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.049134    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 19:21:54.049155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.049143    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 19:21:54.049323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.049271    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4z6tt\""
Sep 10 19:21:54.049433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.049416    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 19:21:54.049567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.049546    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 19:21:54.059597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.059579    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"]
Sep 10 19:21:54.061248 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.061226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrtj\" (UniqueName: \"kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.131406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.131313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:54.147547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.147510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.147547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.147546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.147715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.147565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.147715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.147647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrf9l\" (UniqueName: \"kubernetes.io/projected/a9400e92-698e-412b-a8cd-e22ca15f53c0-kube-api-access-jrf9l\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.147715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.147671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248651 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248731 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrf9l\" (UniqueName: \"kubernetes.io/projected/a9400e92-698e-412b-a8cd-e22ca15f53c0-kube-api-access-jrf9l\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.248844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.248798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-manifests\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.250894 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.250867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9400e92-698e-412b-a8cd-e22ca15f53c0-tmp\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.251135 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.251112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-webhook-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.251208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.251190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a9400e92-698e-412b-a8cd-e22ca15f53c0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.258458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.258434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrf9l\" (UniqueName: \"kubernetes.io/projected/a9400e92-698e-412b-a8cd-e22ca15f53c0-kube-api-access-jrf9l\") pod \"opendatahub-operator-controller-manager-55556678b8-wnn9l\" (UID: \"a9400e92-698e-412b-a8cd-e22ca15f53c0\") " pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.356556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.356521    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:21:54.419619 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.419536    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"]
Sep 10 19:21:54.422837 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:21:54.422805    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0c91e8_1eb1_44c5_9a80_ac0baf0224c9.slice/crio-c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e WatchSource:0}: Error finding container c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e: Status 404 returned error can't find the container with id c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e
Sep 10 19:21:54.550319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.550286    2567 generic.go:356] "Generic (PLEG): container finished" podID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerID="4c7d2d40f79d2a9e662e7c58b43e614ac9c0268ce0eff4862422228221ac180c" exitCode=0
Sep 10 19:21:54.550495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.550373    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c" event={"ID":"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9","Type":"ContainerDied","Data":"4c7d2d40f79d2a9e662e7c58b43e614ac9c0268ce0eff4862422228221ac180c"}
Sep 10 19:21:54.550495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.550426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c" event={"ID":"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9","Type":"ContainerStarted","Data":"c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e"}
Sep 10 19:21:54.647090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:54.647002    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"]
Sep 10 19:21:54.650072 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:21:54.650047    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9400e92_698e_412b_a8cd_e22ca15f53c0.slice/crio-3fe112e4f60b4e841cf2ff1745f2a0d9f06658b5a44eb8b97285b31f0f4d4eb3 WatchSource:0}: Error finding container 3fe112e4f60b4e841cf2ff1745f2a0d9f06658b5a44eb8b97285b31f0f4d4eb3: Status 404 returned error can't find the container with id 3fe112e4f60b4e841cf2ff1745f2a0d9f06658b5a44eb8b97285b31f0f4d4eb3
Sep 10 19:21:55.559871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:55.559832    2567 generic.go:356] "Generic (PLEG): container finished" podID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerID="c11383bfe346296216ba23ec455141134e6d56674fb53e2b8e130629367884aa" exitCode=0
Sep 10 19:21:55.560341 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:55.559947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c" event={"ID":"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9","Type":"ContainerDied","Data":"c11383bfe346296216ba23ec455141134e6d56674fb53e2b8e130629367884aa"}
Sep 10 19:21:55.562263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:55.562235    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l" event={"ID":"a9400e92-698e-412b-a8cd-e22ca15f53c0","Type":"ContainerStarted","Data":"3fe112e4f60b4e841cf2ff1745f2a0d9f06658b5a44eb8b97285b31f0f4d4eb3"}
Sep 10 19:21:56.569951 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:56.569897    2567 generic.go:356] "Generic (PLEG): container finished" podID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerID="30fae2c7429167de2d8e87e3d689676b96aa57f1cdad28acb9c63017d6e83f11" exitCode=0
Sep 10 19:21:56.570360 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:56.569985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c" event={"ID":"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9","Type":"ContainerDied","Data":"30fae2c7429167de2d8e87e3d689676b96aa57f1cdad28acb9c63017d6e83f11"}
Sep 10 19:21:57.794473 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.794448    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:57.880516 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.880488    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsrtj\" (UniqueName: \"kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj\") pod \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") "
Sep 10 19:21:57.880646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.880536    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util\") pod \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") "
Sep 10 19:21:57.880646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.880597    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle\") pod \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\" (UID: \"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9\") "
Sep 10 19:21:57.881489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.881458    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle" (OuterVolumeSpecName: "bundle") pod "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" (UID: "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:57.882551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.882530    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj" (OuterVolumeSpecName: "kube-api-access-zsrtj") pod "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" (UID: "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9"). InnerVolumeSpecName "kube-api-access-zsrtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:21:57.886215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.886183    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util" (OuterVolumeSpecName: "util") pod "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" (UID: "2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:21:57.982295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.982253    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsrtj\" (UniqueName: \"kubernetes.io/projected/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-kube-api-access-zsrtj\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:57.982295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.982289    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-util\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:57.982295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:57.982301    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:21:58.581712 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:58.581686    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c"
Sep 10 19:21:58.581903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:58.581691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0h6l8c" event={"ID":"2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9","Type":"ContainerDied","Data":"c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e"}
Sep 10 19:21:58.581903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:58.581800    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c46770556eb8b6c31c1cc0f68cc61dc0538469e0dccd3294ff8682674efe018e"
Sep 10 19:21:58.583406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:58.583382    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l" event={"ID":"a9400e92-698e-412b-a8cd-e22ca15f53c0","Type":"ContainerStarted","Data":"7d4bf959c4c5bc3520b4fe7b1cbf779c36de3117336ad6c306ee4eeb9c9f8d77"}
Sep 10 19:21:59.590259 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:59.590225    2567 generic.go:356] "Generic (PLEG): container finished" podID="a9400e92-698e-412b-a8cd-e22ca15f53c0" containerID="7d4bf959c4c5bc3520b4fe7b1cbf779c36de3117336ad6c306ee4eeb9c9f8d77" exitCode=0
Sep 10 19:21:59.590664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:21:59.590300    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l" event={"ID":"a9400e92-698e-412b-a8cd-e22ca15f53c0","Type":"ContainerDied","Data":"7d4bf959c4c5bc3520b4fe7b1cbf779c36de3117336ad6c306ee4eeb9c9f8d77"}
Sep 10 19:22:00.597075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:00.596986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l" event={"ID":"a9400e92-698e-412b-a8cd-e22ca15f53c0","Type":"ContainerStarted","Data":"d7859fac7ba058df04b27a538a69d446a13d3d4da36bfc81657ab5ed77978a08"}
Sep 10 19:22:00.597440 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:00.597119    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:22:00.620953 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:00.620895    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l" podStartSLOduration=1.754445999 podStartE2EDuration="6.620879968s" totalImagesPullingTime="3.75271629s" totalInitContainerRuntime="1.113717679s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:21:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:00.61923522 +0000 UTC m=+379.440291375" watchObservedRunningTime="2026-09-10 19:22:00.620879968 +0000 UTC m=+379.441936172"
Sep 10 19:22:07.825139 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825109    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b9dd6998-qv6qz"]
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825456    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="util"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825469    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="util"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825476    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="extract"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825482    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="extract"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825496    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="pull"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825503    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="pull"
Sep 10 19:22:07.825650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.825566    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0c91e8-1eb1-44c5-9a80-ac0baf0224c9" containerName="extract"
Sep 10 19:22:07.828824 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.828807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.840176 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.840152    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b9dd6998-qv6qz"]
Sep 10 19:22:07.863956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.863909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-oauth-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864126 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.863979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r68rs\" (UniqueName: \"kubernetes.io/projected/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-kube-api-access-r68rs\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864126 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.864011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864126 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.864045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.864163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-oauth-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.864195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-trusted-ca-bundle\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.864257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.864227    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-service-ca\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.964807 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.964766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-oauth-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.964807 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.964806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-trusted-ca-bundle\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.964924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-service-ca\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965040    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-oauth-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965070    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r68rs\" (UniqueName: \"kubernetes.io/projected/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-kube-api-access-r68rs\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965744    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-oauth-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965778    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-service-ca\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.965900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.965816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-trusted-ca-bundle\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.967394 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.967364    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-oauth-config\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.967613 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.967593    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-console-serving-cert\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:07.974899 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:07.974878    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r68rs\" (UniqueName: \"kubernetes.io/projected/b6af28e3-88b9-4cd8-a704-6ed31e2515ef-kube-api-access-r68rs\") pod \"console-66b9dd6998-qv6qz\" (UID: \"b6af28e3-88b9-4cd8-a704-6ed31e2515ef\") " pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:08.138613 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:08.138524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:08.418979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:08.418845    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b9dd6998-qv6qz"]
Sep 10 19:22:08.421288 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:22:08.421255    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6af28e3_88b9_4cd8_a704_6ed31e2515ef.slice/crio-6b6a2d6d83af3036a4b1f29bb916254117137d34cde4c925df7b64b7352cfd44 WatchSource:0}: Error finding container 6b6a2d6d83af3036a4b1f29bb916254117137d34cde4c925df7b64b7352cfd44: Status 404 returned error can't find the container with id 6b6a2d6d83af3036a4b1f29bb916254117137d34cde4c925df7b64b7352cfd44
Sep 10 19:22:08.643065 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:08.643030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b9dd6998-qv6qz" event={"ID":"b6af28e3-88b9-4cd8-a704-6ed31e2515ef","Type":"ContainerStarted","Data":"22354c6d1b651ed2538849a4232537ca8e9041ba68aad5d4bccbb1b022ef0ecc"}
Sep 10 19:22:08.643271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:08.643085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b9dd6998-qv6qz" event={"ID":"b6af28e3-88b9-4cd8-a704-6ed31e2515ef","Type":"ContainerStarted","Data":"6b6a2d6d83af3036a4b1f29bb916254117137d34cde4c925df7b64b7352cfd44"}
Sep 10 19:22:08.666965 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:08.666883    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b9dd6998-qv6qz" podStartSLOduration=1.666866613 podStartE2EDuration="1.666866613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:08.664339746 +0000 UTC m=+387.485395880" watchObservedRunningTime="2026-09-10 19:22:08.666866613 +0000 UTC m=+387.487922746"
Sep 10 19:22:11.603824 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:11.603791    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55556678b8-wnn9l"
Sep 10 19:22:12.614527 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.614488    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"]
Sep 10 19:22:12.619386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.619364    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.622809 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622784    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 19:22:12.622979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622784    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 19:22:12.622979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622823    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 19:22:12.622979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622836    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5tpzd\""
Sep 10 19:22:12.622979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622853    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 19:22:12.622979 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.622814    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 19:22:12.634296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.634273    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"]
Sep 10 19:22:12.699022 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.698982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dcd1933a-2b2f-430d-887f-92f1c3d2a573-manager-config\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.699194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.699030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.699194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.699085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-metrics-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.699194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.699149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6grtm\" (UniqueName: \"kubernetes.io/projected/dcd1933a-2b2f-430d-887f-92f1c3d2a573-kube-api-access-6grtm\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.800207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.800173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6grtm\" (UniqueName: \"kubernetes.io/projected/dcd1933a-2b2f-430d-887f-92f1c3d2a573-kube-api-access-6grtm\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.800407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.800242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dcd1933a-2b2f-430d-887f-92f1c3d2a573-manager-config\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.800407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.800269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.800407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.800289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-metrics-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.800916 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.800891    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dcd1933a-2b2f-430d-887f-92f1c3d2a573-manager-config\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.802759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.802738    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.802874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.802856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dcd1933a-2b2f-430d-887f-92f1c3d2a573-metrics-cert\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.810745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.810724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6grtm\" (UniqueName: \"kubernetes.io/projected/dcd1933a-2b2f-430d-887f-92f1c3d2a573-kube-api-access-6grtm\") pod \"lws-controller-manager-77f5df88d-mcqmw\" (UID: \"dcd1933a-2b2f-430d-887f-92f1c3d2a573\") " pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:12.929627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:12.929594    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:13.210257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:13.210228    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"]
Sep 10 19:22:13.212097 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:22:13.212071    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd1933a_2b2f_430d_887f_92f1c3d2a573.slice/crio-7b7bb4341cb3abf1d333c865e58d20a9e4620b3b8b6af767d7dc6486e24a0680 WatchSource:0}: Error finding container 7b7bb4341cb3abf1d333c865e58d20a9e4620b3b8b6af767d7dc6486e24a0680: Status 404 returned error can't find the container with id 7b7bb4341cb3abf1d333c865e58d20a9e4620b3b8b6af767d7dc6486e24a0680
Sep 10 19:22:13.670014 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:13.669979    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw" event={"ID":"dcd1933a-2b2f-430d-887f-92f1c3d2a573","Type":"ContainerStarted","Data":"7b7bb4341cb3abf1d333c865e58d20a9e4620b3b8b6af767d7dc6486e24a0680"}
Sep 10 19:22:15.682610 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:15.682562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw" event={"ID":"dcd1933a-2b2f-430d-887f-92f1c3d2a573","Type":"ContainerStarted","Data":"5ffe57d190450d7bfe791f4294305e469b694d0dd9578c7164c150f6431819b2"}
Sep 10 19:22:15.683006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:15.682670    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:15.702686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:15.702640    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw" podStartSLOduration=1.394915563 podStartE2EDuration="3.702624318s" totalImagesPullingTime="2.307708755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:15.701781571 +0000 UTC m=+394.522837704" watchObservedRunningTime="2026-09-10 19:22:15.702624318 +0000 UTC m=+394.523680450"
Sep 10 19:22:18.139392 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:18.139352    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:18.139809 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:18.139439    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:18.144162 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:18.144141    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:18.702711 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:18.702680    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b9dd6998-qv6qz"
Sep 10 19:22:18.756129 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:18.756100    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:22:24.005407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.005364    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"]
Sep 10 19:22:24.009196 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.009174    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.011562 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.011537    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 19:22:24.011562 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.011557    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9j2pj\""
Sep 10 19:22:24.011740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.011571    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 19:22:24.018973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.018917    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"]
Sep 10 19:22:24.098318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.098280    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbec643-f886-4437-9b3e-5514cccb90f9-tmp\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.098486 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.098326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n7r6\" (UniqueName: \"kubernetes.io/projected/abbec643-f886-4437-9b3e-5514cccb90f9-kube-api-access-7n7r6\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.098486 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.098376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.199817 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.199782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbec643-f886-4437-9b3e-5514cccb90f9-tmp\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.199817 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.199828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n7r6\" (UniqueName: \"kubernetes.io/projected/abbec643-f886-4437-9b3e-5514cccb90f9-kube-api-access-7n7r6\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.200161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.200006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.200161 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:22:24.200119    2567 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 19:22:24.200272 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:22:24.200228    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs podName:abbec643-f886-4437-9b3e-5514cccb90f9 nodeName:}" failed. No retries permitted until 2026-09-10 19:22:24.700205713 +0000 UTC m=+403.521261828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs") pod "kube-auth-proxy-64954c75b6-hn57b" (UID: "abbec643-f886-4437-9b3e-5514cccb90f9") : secret "kube-auth-proxy-tls" not found
Sep 10 19:22:24.202376 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.202349    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/abbec643-f886-4437-9b3e-5514cccb90f9-tmp\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.208683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.208651    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n7r6\" (UniqueName: \"kubernetes.io/projected/abbec643-f886-4437-9b3e-5514cccb90f9-kube-api-access-7n7r6\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.706150 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.706112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.708578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.708558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbec643-f886-4437-9b3e-5514cccb90f9-tls-certs\") pod \"kube-auth-proxy-64954c75b6-hn57b\" (UID: \"abbec643-f886-4437-9b3e-5514cccb90f9\") " pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:24.920261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:24.920224    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"
Sep 10 19:22:25.192334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:25.192303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64954c75b6-hn57b"]
Sep 10 19:22:25.194409 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:22:25.194385    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabbec643_f886_4437_9b3e_5514cccb90f9.slice/crio-aec9a07414ee284a3b56c0b3a866ddf8eadb98c34e14b8bf16eed75859a8066a WatchSource:0}: Error finding container aec9a07414ee284a3b56c0b3a866ddf8eadb98c34e14b8bf16eed75859a8066a: Status 404 returned error can't find the container with id aec9a07414ee284a3b56c0b3a866ddf8eadb98c34e14b8bf16eed75859a8066a
Sep 10 19:22:25.732764 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:25.732729    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b" event={"ID":"abbec643-f886-4437-9b3e-5514cccb90f9","Type":"ContainerStarted","Data":"aec9a07414ee284a3b56c0b3a866ddf8eadb98c34e14b8bf16eed75859a8066a"}
Sep 10 19:22:26.691831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:26.691800    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77f5df88d-mcqmw"
Sep 10 19:22:28.751178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:28.751144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b" event={"ID":"abbec643-f886-4437-9b3e-5514cccb90f9","Type":"ContainerStarted","Data":"285bd8c6f6ad3e2df4dd59b903ed69de7ad6a790470b9c38d351e4aa3ee96411"}
Sep 10 19:22:28.770347 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:28.770297    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64954c75b6-hn57b" podStartSLOduration=3.057517733 podStartE2EDuration="5.770280051s" totalImagesPullingTime="2.712762318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:22:28.769289438 +0000 UTC m=+407.590345572" watchObservedRunningTime="2026-09-10 19:22:28.770280051 +0000 UTC m=+407.591336183"
Sep 10 19:22:43.779893 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:43.779845    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56fc889fdb-2bzsf" podUID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" containerName="console" containerID="cri-o://baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57" gracePeriod=15
Sep 10 19:22:44.027905 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.027874    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:22:44.181281 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181245    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181347    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181386    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181410    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181467    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clw6p\" (UniqueName: \"kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181496    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.181538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181539    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config\") pod \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\" (UID: \"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7\") "
Sep 10 19:22:44.182178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181919    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:44.182178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.181951    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config" (OuterVolumeSpecName: "console-config") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:44.182178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.182143    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:44.182595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.182561    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:22:44.183875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.183845    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:44.184056 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.184032    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:22:44.184132 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.184067    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p" (OuterVolumeSpecName: "kube-api-access-clw6p") pod "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" (UID: "995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7"). InnerVolumeSpecName "kube-api-access-clw6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:22:44.282534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282479    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clw6p\" (UniqueName: \"kubernetes.io/projected/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-kube-api-access-clw6p\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282526    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-oauth-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282536    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-oauth-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282544    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-serving-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282554    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282565    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-trusted-ca-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.282815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.282573    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7-console-config\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:22:44.833254 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.833216    2567 generic.go:356] "Generic (PLEG): container finished" podID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" containerID="baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57" exitCode=0
Sep 10 19:22:44.833733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.833264    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc889fdb-2bzsf"
Sep 10 19:22:44.833733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.833271    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc889fdb-2bzsf" event={"ID":"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7","Type":"ContainerDied","Data":"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"}
Sep 10 19:22:44.833733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.833318    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc889fdb-2bzsf" event={"ID":"995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7","Type":"ContainerDied","Data":"834b620c661b36b74cf8793ac1321008318355d9ac411d520d384d23cb53624a"}
Sep 10 19:22:44.833733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.833342    2567 scope.go:117] "RemoveContainer" containerID="baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"
Sep 10 19:22:44.843194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.843176    2567 scope.go:117] "RemoveContainer" containerID="baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"
Sep 10 19:22:44.843500 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:22:44.843479    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57\": container with ID starting with baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57 not found: ID does not exist" containerID="baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"
Sep 10 19:22:44.843558 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.843510    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57"} err="failed to get container status \"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57\": rpc error: code = NotFound desc = could not find container \"baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57\": container with ID starting with baf0d338c0633a0150411e1bd7854a1220e3deab3c58e518dbddd38497bbff57 not found: ID does not exist"
Sep 10 19:22:44.857814 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.857787    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:22:44.863293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:44.863266    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56fc889fdb-2bzsf"]
Sep 10 19:22:45.702022 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:22:45.701984    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" path="/var/lib/kubelet/pods/995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7/volumes"
Sep 10 19:23:28.602984 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.602942    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"]
Sep 10 19:23:28.603412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.603351    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" containerName="console"
Sep 10 19:23:28.603412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.603367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" containerName="console"
Sep 10 19:23:28.603488 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.603424    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="995f76d5-68f1-4be3-9cf6-0d02b5b1c3e7" containerName="console"
Sep 10 19:23:28.607052 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.607034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.609351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.609327    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 19:23:28.609351 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.609349    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 19:23:28.609528 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.609348    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5snlz\""
Sep 10 19:23:28.614741 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.614723    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"]
Sep 10 19:23:28.670269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.670227    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.670269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.670267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.670487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.670341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bnrq\" (UniqueName: \"kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.771120 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.771080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bnrq\" (UniqueName: \"kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.771457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.771428    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.771580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.771478    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.771751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.771733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.771793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.771769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.779575 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.779543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bnrq\" (UniqueName: \"kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.802556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.802523    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"]
Sep 10 19:23:28.806388 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.806371    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.815401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.815371    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"]
Sep 10 19:23:28.872334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.872228    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dmxx\" (UniqueName: \"kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.872334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.872270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.872334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.872306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.917533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.917500    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:28.973469 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.973442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dmxx\" (UniqueName: \"kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.973596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.973480    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.973596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.973527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.973981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.973955    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.973981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.973973    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:28.983273 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:28.983108    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dmxx\" (UniqueName: \"kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:29.118359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:29.118320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:29.200274 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:29.197586    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"]
Sep 10 19:23:29.392298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:29.392233    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"]
Sep 10 19:23:29.394543 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:23:29.394514    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2db3a55_77b7_4f1a_ba29_5979532fa687.slice/crio-4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7 WatchSource:0}: Error finding container 4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7: Status 404 returned error can't find the container with id 4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7
Sep 10 19:23:30.056108 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.056073    2567 generic.go:356] "Generic (PLEG): container finished" podID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerID="b391c9c5832ee72f0a4482956a6702ddddb201c5a4aa8115dce80f8086933210" exitCode=0
Sep 10 19:23:30.056681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.056163    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl" event={"ID":"8726a4fc-65b2-411f-9d45-3f4aa91a961e","Type":"ContainerDied","Data":"b391c9c5832ee72f0a4482956a6702ddddb201c5a4aa8115dce80f8086933210"}
Sep 10 19:23:30.056681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.056231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl" event={"ID":"8726a4fc-65b2-411f-9d45-3f4aa91a961e","Type":"ContainerStarted","Data":"e324e284dfcfef48372568e425bbde0f853796a5b929c07096877948b8419f4f"}
Sep 10 19:23:30.058167 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.058138    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerID="3c4c2189aefd5a50db2c093a503a86d851c99298b19c58e08c8e8c8b9804d615" exitCode=0
Sep 10 19:23:30.058261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.058216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerDied","Data":"3c4c2189aefd5a50db2c093a503a86d851c99298b19c58e08c8e8c8b9804d615"}
Sep 10 19:23:30.058261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:30.058253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerStarted","Data":"4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7"}
Sep 10 19:23:31.066678 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:31.066643    2567 generic.go:356] "Generic (PLEG): container finished" podID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerID="043f29a2e5a732c66fde3931b118afee0b338792b3bc1c741ade4e1211ea220a" exitCode=0
Sep 10 19:23:31.067099 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:31.066717    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl" event={"ID":"8726a4fc-65b2-411f-9d45-3f4aa91a961e","Type":"ContainerDied","Data":"043f29a2e5a732c66fde3931b118afee0b338792b3bc1c741ade4e1211ea220a"}
Sep 10 19:23:31.068995 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:31.068968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerStarted","Data":"8bc33dad3049f05b9347574d98a414512c498eab23c019dd74b815fbb932df16"}
Sep 10 19:23:32.076836 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:32.076799    2567 generic.go:356] "Generic (PLEG): container finished" podID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerID="367674bb3d219b3cb093ece5b2f0f2177297e4244484ec7f4105aa174d8b945a" exitCode=0
Sep 10 19:23:32.077296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:32.076892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl" event={"ID":"8726a4fc-65b2-411f-9d45-3f4aa91a961e","Type":"ContainerDied","Data":"367674bb3d219b3cb093ece5b2f0f2177297e4244484ec7f4105aa174d8b945a"}
Sep 10 19:23:32.078667 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:32.078647    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerID="8bc33dad3049f05b9347574d98a414512c498eab23c019dd74b815fbb932df16" exitCode=0
Sep 10 19:23:32.078769 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:32.078728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerDied","Data":"8bc33dad3049f05b9347574d98a414512c498eab23c019dd74b815fbb932df16"}
Sep 10 19:23:33.088884 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.088839    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerID="a082e2fe5ad0d2e594bd85ac42e8751786dbd87473d940593ab6f9fab9656523" exitCode=0
Sep 10 19:23:33.089294 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.088918    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerDied","Data":"a082e2fe5ad0d2e594bd85ac42e8751786dbd87473d940593ab6f9fab9656523"}
Sep 10 19:23:33.221515 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.221489    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:33.313296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.313262    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bnrq\" (UniqueName: \"kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq\") pod \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") "
Sep 10 19:23:33.313462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.313321    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle\") pod \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") "
Sep 10 19:23:33.313462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.313365    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util\") pod \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\" (UID: \"8726a4fc-65b2-411f-9d45-3f4aa91a961e\") "
Sep 10 19:23:33.313896 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.313868    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle" (OuterVolumeSpecName: "bundle") pod "8726a4fc-65b2-411f-9d45-3f4aa91a961e" (UID: "8726a4fc-65b2-411f-9d45-3f4aa91a961e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:23:33.315544 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.315521    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq" (OuterVolumeSpecName: "kube-api-access-4bnrq") pod "8726a4fc-65b2-411f-9d45-3f4aa91a961e" (UID: "8726a4fc-65b2-411f-9d45-3f4aa91a961e"). InnerVolumeSpecName "kube-api-access-4bnrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:33.321997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.321954    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util" (OuterVolumeSpecName: "util") pod "8726a4fc-65b2-411f-9d45-3f4aa91a961e" (UID: "8726a4fc-65b2-411f-9d45-3f4aa91a961e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:23:33.414215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.414180    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bnrq\" (UniqueName: \"kubernetes.io/projected/8726a4fc-65b2-411f-9d45-3f4aa91a961e-kube-api-access-4bnrq\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:33.414215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.414216    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:33.414426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:33.414226    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8726a4fc-65b2-411f-9d45-3f4aa91a961e-util\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:34.096268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.096178    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl"
Sep 10 19:23:34.096268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.096209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7b2gtl" event={"ID":"8726a4fc-65b2-411f-9d45-3f4aa91a961e","Type":"ContainerDied","Data":"e324e284dfcfef48372568e425bbde0f853796a5b929c07096877948b8419f4f"}
Sep 10 19:23:34.096268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.096254    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e324e284dfcfef48372568e425bbde0f853796a5b929c07096877948b8419f4f"
Sep 10 19:23:34.232145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.232119    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:34.322700 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.322645    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util\") pod \"a2db3a55-77b7-4f1a-ba29-5979532fa687\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") "
Sep 10 19:23:34.322887 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.322744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle\") pod \"a2db3a55-77b7-4f1a-ba29-5979532fa687\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") "
Sep 10 19:23:34.322887 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.322804    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dmxx\" (UniqueName: \"kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx\") pod \"a2db3a55-77b7-4f1a-ba29-5979532fa687\" (UID: \"a2db3a55-77b7-4f1a-ba29-5979532fa687\") "
Sep 10 19:23:34.323591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.323559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle" (OuterVolumeSpecName: "bundle") pod "a2db3a55-77b7-4f1a-ba29-5979532fa687" (UID: "a2db3a55-77b7-4f1a-ba29-5979532fa687"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:23:34.324983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.324952    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx" (OuterVolumeSpecName: "kube-api-access-4dmxx") pod "a2db3a55-77b7-4f1a-ba29-5979532fa687" (UID: "a2db3a55-77b7-4f1a-ba29-5979532fa687"). InnerVolumeSpecName "kube-api-access-4dmxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:23:34.328591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.328567    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util" (OuterVolumeSpecName: "util") pod "a2db3a55-77b7-4f1a-ba29-5979532fa687" (UID: "a2db3a55-77b7-4f1a-ba29-5979532fa687"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:23:34.424280 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.424243    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dmxx\" (UniqueName: \"kubernetes.io/projected/a2db3a55-77b7-4f1a-ba29-5979532fa687-kube-api-access-4dmxx\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:34.424280 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.424274    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-util\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:34.424280 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:34.424285    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2db3a55-77b7-4f1a-ba29-5979532fa687-bundle\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:23:35.104569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:35.104536    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw"
Sep 10 19:23:35.105018 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:35.104531    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847agn2nw" event={"ID":"a2db3a55-77b7-4f1a-ba29-5979532fa687","Type":"ContainerDied","Data":"4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7"}
Sep 10 19:23:35.105018 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:35.104665    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e3c6c536bc04fb2c085b375512801c5894b42e66507dc0f3525c109ac575fe7"
Sep 10 19:23:56.707261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707229    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"]
Sep 10 19:23:56.707842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707784    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="util"
Sep 10 19:23:56.707842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707805    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="util"
Sep 10 19:23:56.707842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707826    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="pull"
Sep 10 19:23:56.707842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707834    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="pull"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707847    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="util"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707855    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="util"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707874    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="extract"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707883    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="extract"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707896    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="extract"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707904    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="extract"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707915    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="pull"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.707923    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="pull"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.708057    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2db3a55-77b7-4f1a-ba29-5979532fa687" containerName="extract"
Sep 10 19:23:56.708088 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.708070    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8726a4fc-65b2-411f-9d45-3f4aa91a961e" containerName="extract"
Sep 10 19:23:56.712508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.712487    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.715260 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.715045    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 19:23:56.715260 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.715096    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 19:23:56.715779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.715757    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 19:23:56.715888 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.715757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k4gv2\""
Sep 10 19:23:56.716327 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.715805    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 19:23:56.718134 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.718112    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"]
Sep 10 19:23:56.817556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.817521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.817732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.817563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llb8v\" (UniqueName: \"kubernetes.io/projected/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-kube-api-access-llb8v\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.817732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.817659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.918435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.918398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.918597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.918525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.918597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.918575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llb8v\" (UniqueName: \"kubernetes.io/projected/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-kube-api-access-llb8v\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.919186 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.919163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.920801 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.920782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:56.927676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:56.927647    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llb8v\" (UniqueName: \"kubernetes.io/projected/316edcbe-4bcd-4999-aed5-9b87dc2d82f8-kube-api-access-llb8v\") pod \"kuadrant-console-plugin-fbdd4f445-zgwmt\" (UID: \"316edcbe-4bcd-4999-aed5-9b87dc2d82f8\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:57.022980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:57.022880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"
Sep 10 19:23:57.301926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:57.301895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt"]
Sep 10 19:23:57.303688 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:23:57.303656    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod316edcbe_4bcd_4999_aed5_9b87dc2d82f8.slice/crio-4f021ee19a63298e8393ad2703d8742d8c276eacb1d6a78255410b71b5a48b68 WatchSource:0}: Error finding container 4f021ee19a63298e8393ad2703d8742d8c276eacb1d6a78255410b71b5a48b68: Status 404 returned error can't find the container with id 4f021ee19a63298e8393ad2703d8742d8c276eacb1d6a78255410b71b5a48b68
Sep 10 19:23:58.237756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:23:58.237674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt" event={"ID":"316edcbe-4bcd-4999-aed5-9b87dc2d82f8","Type":"ContainerStarted","Data":"4f021ee19a63298e8393ad2703d8742d8c276eacb1d6a78255410b71b5a48b68"}
Sep 10 19:24:02.264077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:02.263970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt" event={"ID":"316edcbe-4bcd-4999-aed5-9b87dc2d82f8","Type":"ContainerStarted","Data":"e1a4d4d9d5a6b8a2113bad81c48b86858ab9db3c90052da8cd53730561b7141d"}
Sep 10 19:24:02.283179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:02.283123    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zgwmt" podStartSLOduration=1.6549866180000001 podStartE2EDuration="6.283106031s" totalImagesPullingTime="4.628119413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:02.281360551 +0000 UTC m=+501.102416685" watchObservedRunningTime="2026-09-10 19:24:02.283106031 +0000 UTC m=+501.104162164"
Sep 10 19:24:52.017968 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.017879    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:24:52.021593 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.021571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.024134 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.024111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 19:24:52.032155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.032133    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:24:52.056944 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.056910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:24:52.088842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.088816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e8ec980-8f03-4e98-9c8d-c935cab80af7-config-file\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.088985 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.088860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbww6\" (UniqueName: \"kubernetes.io/projected/7e8ec980-8f03-4e98-9c8d-c935cab80af7-kube-api-access-pbww6\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.190089 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.190055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e8ec980-8f03-4e98-9c8d-c935cab80af7-config-file\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.190253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.190099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbww6\" (UniqueName: \"kubernetes.io/projected/7e8ec980-8f03-4e98-9c8d-c935cab80af7-kube-api-access-pbww6\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.190677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.190657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7e8ec980-8f03-4e98-9c8d-c935cab80af7-config-file\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.199192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.199157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbww6\" (UniqueName: \"kubernetes.io/projected/7e8ec980-8f03-4e98-9c8d-c935cab80af7-kube-api-access-pbww6\") pod \"limitador-limitador-84994f7ffc-ls7r4\" (UID: \"7e8ec980-8f03-4e98-9c8d-c935cab80af7\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.332770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.332683    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:52.429991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.429955    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:52.433589 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.433571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:52.436074 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.436046    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vfvhr\""
Sep 10 19:24:52.442378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.442356    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:52.493483 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.493445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxk88\" (UniqueName: \"kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88\") pod \"authorino-7bcfd8f9cd-l29fd\" (UID: \"539dc55f-8878-4bb3-a584-a176be8fb191\") " pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:52.594685 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.594608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxk88\" (UniqueName: \"kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88\") pod \"authorino-7bcfd8f9cd-l29fd\" (UID: \"539dc55f-8878-4bb3-a584-a176be8fb191\") " pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:52.603624 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.603603    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:24:52.605427 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.605397    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxk88\" (UniqueName: \"kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88\") pod \"authorino-7bcfd8f9cd-l29fd\" (UID: \"539dc55f-8878-4bb3-a584-a176be8fb191\") " pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:52.605549 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:24:52.605527    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8ec980_8f03_4e98_9c8d_c935cab80af7.slice/crio-f1eb1fb975e37d15d34d354ae646f58b0bd25a72e6d89665e48105d0192a002c WatchSource:0}: Error finding container f1eb1fb975e37d15d34d354ae646f58b0bd25a72e6d89665e48105d0192a002c: Status 404 returned error can't find the container with id f1eb1fb975e37d15d34d354ae646f58b0bd25a72e6d89665e48105d0192a002c
Sep 10 19:24:52.663633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.663598    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:24:52.667084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.667069    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:24:52.676374 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.676347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:24:52.696068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.696028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2krnc\" (UniqueName: \"kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc\") pod \"authorino-59c974bd48-v9j4n\" (UID: \"6b9314f4-2d2a-460a-ae73-077e80056c52\") " pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:24:52.745333 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.745303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:52.798250 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.797811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2krnc\" (UniqueName: \"kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc\") pod \"authorino-59c974bd48-v9j4n\" (UID: \"6b9314f4-2d2a-460a-ae73-077e80056c52\") " pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:24:52.807387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.807358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2krnc\" (UniqueName: \"kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc\") pod \"authorino-59c974bd48-v9j4n\" (UID: \"6b9314f4-2d2a-460a-ae73-077e80056c52\") " pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:24:52.976976 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:52.976945    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:24:53.015859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:53.015822    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:53.019757 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:24:53.019721    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod539dc55f_8878_4bb3_a584_a176be8fb191.slice/crio-c63bb4932ca2d09d3f9e5627ac547b217db787a2d6f69f63d6ecb26136f089ef WatchSource:0}: Error finding container c63bb4932ca2d09d3f9e5627ac547b217db787a2d6f69f63d6ecb26136f089ef: Status 404 returned error can't find the container with id c63bb4932ca2d09d3f9e5627ac547b217db787a2d6f69f63d6ecb26136f089ef
Sep 10 19:24:53.253718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:53.253678    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:24:53.256466 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:24:53.256435    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b9314f4_2d2a_460a_ae73_077e80056c52.slice/crio-ac98e0869345476bf15f74a125519722c4dae952d1ea541ef8e1fecc34092454 WatchSource:0}: Error finding container ac98e0869345476bf15f74a125519722c4dae952d1ea541ef8e1fecc34092454: Status 404 returned error can't find the container with id ac98e0869345476bf15f74a125519722c4dae952d1ea541ef8e1fecc34092454
Sep 10 19:24:53.526426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:53.526335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v9j4n" event={"ID":"6b9314f4-2d2a-460a-ae73-077e80056c52","Type":"ContainerStarted","Data":"ac98e0869345476bf15f74a125519722c4dae952d1ea541ef8e1fecc34092454"}
Sep 10 19:24:53.527827 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:53.527794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4" event={"ID":"7e8ec980-8f03-4e98-9c8d-c935cab80af7","Type":"ContainerStarted","Data":"f1eb1fb975e37d15d34d354ae646f58b0bd25a72e6d89665e48105d0192a002c"}
Sep 10 19:24:53.529106 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:53.529074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" event={"ID":"539dc55f-8878-4bb3-a584-a176be8fb191","Type":"ContainerStarted","Data":"c63bb4932ca2d09d3f9e5627ac547b217db787a2d6f69f63d6ecb26136f089ef"}
Sep 10 19:24:56.553785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.553746    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4" event={"ID":"7e8ec980-8f03-4e98-9c8d-c935cab80af7","Type":"ContainerStarted","Data":"7ffd7e89f4d56695bdaa3866ea0463da7fe822424af6c4cb6fa303a66dd6b97c"}
Sep 10 19:24:56.554245 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.553844    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:24:56.555427 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.555403    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" event={"ID":"539dc55f-8878-4bb3-a584-a176be8fb191","Type":"ContainerStarted","Data":"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"}
Sep 10 19:24:56.556824 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.556804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v9j4n" event={"ID":"6b9314f4-2d2a-460a-ae73-077e80056c52","Type":"ContainerStarted","Data":"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"}
Sep 10 19:24:56.573186 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.573139    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4" podStartSLOduration=1.4760051650000001 podStartE2EDuration="4.573124674s" totalImagesPullingTime="3.097119509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:56.571001986 +0000 UTC m=+555.392058118" watchObservedRunningTime="2026-09-10 19:24:56.573124674 +0000 UTC m=+555.394180807"
Sep 10 19:24:56.586772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.586723    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" podStartSLOduration=1.90279447 podStartE2EDuration="4.586708098s" totalImagesPullingTime="2.683913628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:56.585490023 +0000 UTC m=+555.406546156" watchObservedRunningTime="2026-09-10 19:24:56.586708098 +0000 UTC m=+555.407764231"
Sep 10 19:24:56.602153 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.602090    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-v9j4n" podStartSLOduration=2.144661413 podStartE2EDuration="4.602072147s" totalImagesPullingTime="2.457410734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:24:56.598599918 +0000 UTC m=+555.419656080" watchObservedRunningTime="2026-09-10 19:24:56.602072147 +0000 UTC m=+555.423128280"
Sep 10 19:24:56.633023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:56.632984    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:58.567703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:58.567663    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" podUID="539dc55f-8878-4bb3-a584-a176be8fb191" containerName="authorino" containerID="cri-o://cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a" gracePeriod=30
Sep 10 19:24:58.816928 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:58.816899    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:58.856687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:58.856610    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxk88\" (UniqueName: \"kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88\") pod \"539dc55f-8878-4bb3-a584-a176be8fb191\" (UID: \"539dc55f-8878-4bb3-a584-a176be8fb191\") "
Sep 10 19:24:58.858718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:58.858692    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88" (OuterVolumeSpecName: "kube-api-access-pxk88") pod "539dc55f-8878-4bb3-a584-a176be8fb191" (UID: "539dc55f-8878-4bb3-a584-a176be8fb191"). InnerVolumeSpecName "kube-api-access-pxk88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:24:58.957783 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:58.957748    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxk88\" (UniqueName: \"kubernetes.io/projected/539dc55f-8878-4bb3-a584-a176be8fb191-kube-api-access-pxk88\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:24:59.574089 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.574057    2567 generic.go:356] "Generic (PLEG): container finished" podID="539dc55f-8878-4bb3-a584-a176be8fb191" containerID="cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a" exitCode=0
Sep 10 19:24:59.574484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.574098    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd"
Sep 10 19:24:59.574484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.574141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" event={"ID":"539dc55f-8878-4bb3-a584-a176be8fb191","Type":"ContainerDied","Data":"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"}
Sep 10 19:24:59.574484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.574182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-l29fd" event={"ID":"539dc55f-8878-4bb3-a584-a176be8fb191","Type":"ContainerDied","Data":"c63bb4932ca2d09d3f9e5627ac547b217db787a2d6f69f63d6ecb26136f089ef"}
Sep 10 19:24:59.574484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.574204    2567 scope.go:117] "RemoveContainer" containerID="cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"
Sep 10 19:24:59.583493 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.583478    2567 scope.go:117] "RemoveContainer" containerID="cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"
Sep 10 19:24:59.583752 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:24:59.583735    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a\": container with ID starting with cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a not found: ID does not exist" containerID="cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"
Sep 10 19:24:59.583796 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.583761    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a"} err="failed to get container status \"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a\": rpc error: code = NotFound desc = could not find container \"cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a\": container with ID starting with cccec596db79bb0c43ac83ebbca592a8757881bed51ae1ff79d048e47015a88a not found: ID does not exist"
Sep 10 19:24:59.596504 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.596480    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:59.600749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.600715    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-l29fd"]
Sep 10 19:24:59.702718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:24:59.702682    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="539dc55f-8878-4bb3-a584-a176be8fb191" path="/var/lib/kubelet/pods/539dc55f-8878-4bb3-a584-a176be8fb191/volumes"
Sep 10 19:25:07.563759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:07.563721    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"
Sep 10 19:25:21.609000 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.608961    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-hc56s"]
Sep 10 19:25:21.609538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.609517    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="539dc55f-8878-4bb3-a584-a176be8fb191" containerName="authorino"
Sep 10 19:25:21.609624 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.609541    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="539dc55f-8878-4bb3-a584-a176be8fb191" containerName="authorino"
Sep 10 19:25:21.609679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.609650    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="539dc55f-8878-4bb3-a584-a176be8fb191" containerName="authorino"
Sep 10 19:25:21.612693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.612672    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:21.616192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.616168    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-hc56s"]
Sep 10 19:25:21.741985 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.741954    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-hc56s"]
Sep 10 19:25:21.742248 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:25:21.742230    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-w875j], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-hc56s" podUID="c0208b2f-648b-4a95-b1c3-3242d906e918"
Sep 10 19:25:21.767676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.767639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w875j\" (UniqueName: \"kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j\") pod \"authorino-557f876f69-hc56s\" (UID: \"c0208b2f-648b-4a95-b1c3-3242d906e918\") " pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:21.769761 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.769736    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-flnqq"]
Sep 10 19:25:21.773337 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.773320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.775498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.775475    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 19:25:21.782574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.782551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-flnqq"]
Sep 10 19:25:21.845846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.845811    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-flnqq"]
Sep 10 19:25:21.846134 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:25:21.846110    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-b88rv tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-flnqq" podUID="55b35225-f0e1-43ea-bc26-08d0b8de36ad"
Sep 10 19:25:21.869043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.868949    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b88rv\" (UniqueName: \"kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.869043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.869005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.869292 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.869065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w875j\" (UniqueName: \"kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j\") pod \"authorino-557f876f69-hc56s\" (UID: \"c0208b2f-648b-4a95-b1c3-3242d906e918\") " pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:21.879815 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.879789    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:25:21.883225 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.883208    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:21.884580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.884560    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w875j\" (UniqueName: \"kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j\") pod \"authorino-557f876f69-hc56s\" (UID: \"c0208b2f-648b-4a95-b1c3-3242d906e918\") " pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:21.892854 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.892830    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:25:21.969846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.969808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6drhc\" (UniqueName: \"kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:21.970081 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.969857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.970081 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.970028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:21.970081 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.970076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b88rv\" (UniqueName: \"kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.972349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.972329    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:21.980809 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:21.980782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b88rv\" (UniqueName: \"kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv\") pod \"authorino-6d4dfc649f-flnqq\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") " pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:22.071294 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.071257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6drhc\" (UniqueName: \"kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:22.071529 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.071352    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:22.073700 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.073672    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:22.080577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.080550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6drhc\" (UniqueName: \"kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc\") pod \"authorino-758dd97fdb-mhmx8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") " pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:22.201023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.200991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:25:22.472825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.472803    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:25:22.475057 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:25:22.475030    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d98edcb_003f_4524_b6d5_70c9b4b325c8.slice/crio-40f0f21e8eac40ef54af9a4caa52c3230628502071cab5a801a9eeba07101744 WatchSource:0}: Error finding container 40f0f21e8eac40ef54af9a4caa52c3230628502071cab5a801a9eeba07101744: Status 404 returned error can't find the container with id 40f0f21e8eac40ef54af9a4caa52c3230628502071cab5a801a9eeba07101744
Sep 10 19:25:22.698609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.698572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" event={"ID":"5d98edcb-003f-4524-b6d5-70c9b4b325c8","Type":"ContainerStarted","Data":"40f0f21e8eac40ef54af9a4caa52c3230628502071cab5a801a9eeba07101744"}
Sep 10 19:25:22.698609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.698593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:22.698609 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.698602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:22.707096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.707075    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:22.709116 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.709102    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:22.879204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.879111    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w875j\" (UniqueName: \"kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j\") pod \"c0208b2f-648b-4a95-b1c3-3242d906e918\" (UID: \"c0208b2f-648b-4a95-b1c3-3242d906e918\") "
Sep 10 19:25:22.879204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.879156    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b88rv\" (UniqueName: \"kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv\") pod \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") "
Sep 10 19:25:22.879439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.879271    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert\") pod \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\" (UID: \"55b35225-f0e1-43ea-bc26-08d0b8de36ad\") "
Sep 10 19:25:22.881228 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.881203    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "55b35225-f0e1-43ea-bc26-08d0b8de36ad" (UID: "55b35225-f0e1-43ea-bc26-08d0b8de36ad"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:25:22.881344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.881243    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j" (OuterVolumeSpecName: "kube-api-access-w875j") pod "c0208b2f-648b-4a95-b1c3-3242d906e918" (UID: "c0208b2f-648b-4a95-b1c3-3242d906e918"). InnerVolumeSpecName "kube-api-access-w875j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:25:22.881344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.881255    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv" (OuterVolumeSpecName: "kube-api-access-b88rv") pod "55b35225-f0e1-43ea-bc26-08d0b8de36ad" (UID: "55b35225-f0e1-43ea-bc26-08d0b8de36ad"). InnerVolumeSpecName "kube-api-access-b88rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:25:22.980765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.980733    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55b35225-f0e1-43ea-bc26-08d0b8de36ad-tls-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:25:22.980765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.980761    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w875j\" (UniqueName: \"kubernetes.io/projected/c0208b2f-648b-4a95-b1c3-3242d906e918-kube-api-access-w875j\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:25:22.980765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:22.980771    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b88rv\" (UniqueName: \"kubernetes.io/projected/55b35225-f0e1-43ea-bc26-08d0b8de36ad-kube-api-access-b88rv\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:25:23.712959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.712915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-hc56s"
Sep 10 19:25:23.712959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.712915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" event={"ID":"5d98edcb-003f-4524-b6d5-70c9b4b325c8","Type":"ContainerStarted","Data":"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"}
Sep 10 19:25:23.713429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.713218    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-flnqq"
Sep 10 19:25:23.729372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.729324    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" podStartSLOduration=1.777915314 podStartE2EDuration="2.729308405s" totalImagesPullingTime="951.393091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:25:23.727589999 +0000 UTC m=+582.548646133" watchObservedRunningTime="2026-09-10 19:25:23.729308405 +0000 UTC m=+582.550364542"
Sep 10 19:25:23.750780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.750751    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-hc56s"]
Sep 10 19:25:23.758555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.758518    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:25:23.758732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.758712    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-v9j4n" podUID="6b9314f4-2d2a-460a-ae73-077e80056c52" containerName="authorino" containerID="cri-o://471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d" gracePeriod=30
Sep 10 19:25:23.762008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.761980    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-hc56s"]
Sep 10 19:25:23.785418 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.785387    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-flnqq"]
Sep 10 19:25:23.790403 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:23.790376    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-flnqq"]
Sep 10 19:25:24.004457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.004431    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:25:24.191694 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.191653    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2krnc\" (UniqueName: \"kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc\") pod \"6b9314f4-2d2a-460a-ae73-077e80056c52\" (UID: \"6b9314f4-2d2a-460a-ae73-077e80056c52\") "
Sep 10 19:25:24.193763 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.193737    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc" (OuterVolumeSpecName: "kube-api-access-2krnc") pod "6b9314f4-2d2a-460a-ae73-077e80056c52" (UID: "6b9314f4-2d2a-460a-ae73-077e80056c52"). InnerVolumeSpecName "kube-api-access-2krnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:25:24.293093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.293007    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2krnc\" (UniqueName: \"kubernetes.io/projected/6b9314f4-2d2a-460a-ae73-077e80056c52-kube-api-access-2krnc\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:25:24.719104 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.719071    2567 generic.go:356] "Generic (PLEG): container finished" podID="6b9314f4-2d2a-460a-ae73-077e80056c52" containerID="471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d" exitCode=0
Sep 10 19:25:24.719561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.719111    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v9j4n"
Sep 10 19:25:24.719561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.719158    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v9j4n" event={"ID":"6b9314f4-2d2a-460a-ae73-077e80056c52","Type":"ContainerDied","Data":"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"}
Sep 10 19:25:24.719561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.719195    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v9j4n" event={"ID":"6b9314f4-2d2a-460a-ae73-077e80056c52","Type":"ContainerDied","Data":"ac98e0869345476bf15f74a125519722c4dae952d1ea541ef8e1fecc34092454"}
Sep 10 19:25:24.719561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.719214    2567 scope.go:117] "RemoveContainer" containerID="471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"
Sep 10 19:25:24.728432 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.728415    2567 scope.go:117] "RemoveContainer" containerID="471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"
Sep 10 19:25:24.728723 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:25:24.728705    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d\": container with ID starting with 471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d not found: ID does not exist" containerID="471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"
Sep 10 19:25:24.728776 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.728733    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d"} err="failed to get container status \"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d\": rpc error: code = NotFound desc = could not find container \"471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d\": container with ID starting with 471ed85e9463db1f02a67411c7aff60f5860ec01f91f70211291e1985338126d not found: ID does not exist"
Sep 10 19:25:24.741856 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.741827    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:25:24.743822 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:24.743802    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v9j4n"]
Sep 10 19:25:25.702419 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:25.702383    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b35225-f0e1-43ea-bc26-08d0b8de36ad" path="/var/lib/kubelet/pods/55b35225-f0e1-43ea-bc26-08d0b8de36ad/volumes"
Sep 10 19:25:25.702696 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:25.702677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9314f4-2d2a-460a-ae73-077e80056c52" path="/var/lib/kubelet/pods/6b9314f4-2d2a-460a-ae73-077e80056c52/volumes"
Sep 10 19:25:25.703054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:25.703037    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0208b2f-648b-4a95-b1c3-3242d906e918" path="/var/lib/kubelet/pods/c0208b2f-648b-4a95-b1c3-3242d906e918/volumes"
Sep 10 19:25:41.648601 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:41.648573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:25:41.649126 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:41.648778    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:25:41.656082 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:41.656059    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:25:41.656276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:25:41.656259    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:26:05.706370 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:26:05.706294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:26:28.766300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:26:28.766263    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:26:44.629703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:26:44.629668    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:26:52.731689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:26:52.731656    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:27:03.146458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:03.146415    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:27:13.158530 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:13.158491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:27:19.617872 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.617838    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"]
Sep 10 19:27:19.618399 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.618239    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9314f4-2d2a-460a-ae73-077e80056c52" containerName="authorino"
Sep 10 19:27:19.618399 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.618251    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9314f4-2d2a-460a-ae73-077e80056c52" containerName="authorino"
Sep 10 19:27:19.618399 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.618308    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9314f4-2d2a-460a-ae73-077e80056c52" containerName="authorino"
Sep 10 19:27:19.621230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.621209    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.623894 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.623873    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 19:27:19.624426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.624407    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 19:27:19.624532 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.624431    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 19:27:19.624532 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.624508    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-28lpp\""
Sep 10 19:27:19.635724 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.635698    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"]
Sep 10 19:27:19.708103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708071    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9j4\" (UniqueName: \"kubernetes.io/projected/86c35c34-c738-4649-a127-1a4be42a29d6-kube-api-access-bc9j4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.708103 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.708311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.708311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708265    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c35c34-c738-4649-a127-1a4be42a29d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.708311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.708416 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.708339    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.809666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.809631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9j4\" (UniqueName: \"kubernetes.io/projected/86c35c34-c738-4649-a127-1a4be42a29d6-kube-api-access-bc9j4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.809666 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.809667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.809910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.809806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c35c34-c738-4649-a127-1a4be42a29d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810604 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.810676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.810628    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.812030 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.812011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86c35c34-c738-4649-a127-1a4be42a29d6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.812625 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.812605    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86c35c34-c738-4649-a127-1a4be42a29d6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.821143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.821119    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9j4\" (UniqueName: \"kubernetes.io/projected/86c35c34-c738-4649-a127-1a4be42a29d6-kube-api-access-bc9j4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vd7tm\" (UID: \"86c35c34-c738-4649-a127-1a4be42a29d6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:19.937038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:19.936998    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:20.214282 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:20.214233    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"]
Sep 10 19:27:20.216544 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:27:20.216520    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c35c34_c738_4649_a127_1a4be42a29d6.slice/crio-82c5f2065ebcd892a82e434f84a62bed143b8d5d8322fd1c81ae0ff7675cdedd WatchSource:0}: Error finding container 82c5f2065ebcd892a82e434f84a62bed143b8d5d8322fd1c81ae0ff7675cdedd: Status 404 returned error can't find the container with id 82c5f2065ebcd892a82e434f84a62bed143b8d5d8322fd1c81ae0ff7675cdedd
Sep 10 19:27:20.217765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:20.217745    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:27:20.327338 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:20.327301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm" event={"ID":"86c35c34-c738-4649-a127-1a4be42a29d6","Type":"ContainerStarted","Data":"82c5f2065ebcd892a82e434f84a62bed143b8d5d8322fd1c81ae0ff7675cdedd"}
Sep 10 19:27:21.334261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:21.334224    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm" event={"ID":"86c35c34-c738-4649-a127-1a4be42a29d6","Type":"ContainerStarted","Data":"d71aba6d41dd6d738eca2a0336075ae7f91a8276f79185db429deae6b2375fc4"}
Sep 10 19:27:21.357929 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:21.357866    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm" podStartSLOduration=1.361603767 podStartE2EDuration="2.357849161s" totalImagesPullingTime="996.245394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:21.355286443 +0000 UTC m=+700.176342576" watchObservedRunningTime="2026-09-10 19:27:21.357849161 +0000 UTC m=+700.178905294"
Sep 10 19:27:21.734698 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:21.734665    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:27:29.938195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:29.938160    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:29.938680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:29.938220    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:29.950679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:29.950656    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:30.395266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:30.395241    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vd7tm"
Sep 10 19:27:37.037395 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.037301    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"]
Sep 10 19:27:37.041162 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.041139    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.043535 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.043515    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 19:27:37.050737 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.050709    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"]
Sep 10 19:27:37.176615 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176572    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.176772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.176772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.176772 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54r8\" (UniqueName: \"kubernetes.io/projected/77258ff2-b9c3-4188-a810-a33588c6acc5-kube-api-access-h54r8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.176969 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176828    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.176969 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.176846    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77258ff2-b9c3-4188-a810-a33588c6acc5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278042    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278094    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278139    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54r8\" (UniqueName: \"kubernetes.io/projected/77258ff2-b9c3-4188-a810-a33588c6acc5-kube-api-access-h54r8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278267    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77258ff2-b9c3-4188-a810-a33588c6acc5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278629 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.278629 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.278602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.280347 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.280316    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77258ff2-b9c3-4188-a810-a33588c6acc5-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.280650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.280631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77258ff2-b9c3-4188-a810-a33588c6acc5-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.289810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.289750    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54r8\" (UniqueName: \"kubernetes.io/projected/77258ff2-b9c3-4188-a810-a33588c6acc5-kube-api-access-h54r8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6lb24\" (UID: \"77258ff2-b9c3-4188-a810-a33588c6acc5\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.351501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.351462    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:37.630758 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:37.630731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"]
Sep 10 19:27:37.632502 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:27:37.632473    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77258ff2_b9c3_4188_a810_a33588c6acc5.slice/crio-de771a07f18eab3a5c2b476e39c02e00113059e4d14453dc96cf64b487947cc6 WatchSource:0}: Error finding container de771a07f18eab3a5c2b476e39c02e00113059e4d14453dc96cf64b487947cc6: Status 404 returned error can't find the container with id de771a07f18eab3a5c2b476e39c02e00113059e4d14453dc96cf64b487947cc6
Sep 10 19:27:38.434313 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:38.434273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24" event={"ID":"77258ff2-b9c3-4188-a810-a33588c6acc5","Type":"ContainerStarted","Data":"d12968ac854ba17aadce2194895de46469c232d20cf7ed083fd36c6ee464c400"}
Sep 10 19:27:38.434313 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:38.434325    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24" event={"ID":"77258ff2-b9c3-4188-a810-a33588c6acc5","Type":"ContainerStarted","Data":"de771a07f18eab3a5c2b476e39c02e00113059e4d14453dc96cf64b487947cc6"}
Sep 10 19:27:38.456092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:38.456039    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24" podStartSLOduration=1.273869071 podStartE2EDuration="1.456021701s" totalImagesPullingTime="182.15263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:27:38.453839215 +0000 UTC m=+717.274895360" watchObservedRunningTime="2026-09-10 19:27:38.456021701 +0000 UTC m=+717.277077828"
Sep 10 19:27:39.327318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:39.327278    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:27:47.351792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:47.351741    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:47.352253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:47.351811    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:47.364409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:47.364380    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:27:47.496161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:27:47.496129    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6lb24"
Sep 10 19:28:59.351095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.351064    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79d67c4d47-5928k"]
Sep 10 19:28:59.354561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.354541    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.364464 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.364437    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d67c4d47-5928k"]
Sep 10 19:28:59.425900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.425867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wswtc\" (UniqueName: \"kubernetes.io/projected/f7507dba-9d78-49e1-a58a-2662b355f070-kube-api-access-wswtc\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.426105 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.425997    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7507dba-9d78-49e1-a58a-2662b355f070-tls-cert\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.527128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.527097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wswtc\" (UniqueName: \"kubernetes.io/projected/f7507dba-9d78-49e1-a58a-2662b355f070-kube-api-access-wswtc\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.527309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.527196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7507dba-9d78-49e1-a58a-2662b355f070-tls-cert\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.529736 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.529702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f7507dba-9d78-49e1-a58a-2662b355f070-tls-cert\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.535453 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.535428    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wswtc\" (UniqueName: \"kubernetes.io/projected/f7507dba-9d78-49e1-a58a-2662b355f070-kube-api-access-wswtc\") pod \"authorino-79d67c4d47-5928k\" (UID: \"f7507dba-9d78-49e1-a58a-2662b355f070\") " pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.664482 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.664447    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79d67c4d47-5928k"
Sep 10 19:28:59.936874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:28:59.936848    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79d67c4d47-5928k"]
Sep 10 19:28:59.939237 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:28:59.939210    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7507dba_9d78_49e1_a58a_2662b355f070.slice/crio-bb8a206ecdd1b38eeceb59204494a0ae6fe16879b53c39f4e54996192083563e WatchSource:0}: Error finding container bb8a206ecdd1b38eeceb59204494a0ae6fe16879b53c39f4e54996192083563e: Status 404 returned error can't find the container with id bb8a206ecdd1b38eeceb59204494a0ae6fe16879b53c39f4e54996192083563e
Sep 10 19:29:00.898907 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:00.898844    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d67c4d47-5928k" event={"ID":"f7507dba-9d78-49e1-a58a-2662b355f070","Type":"ContainerStarted","Data":"1500a247e4c7a112ac812b1d8331b97475bedc6b30f55b4539701bd53b800140"}
Sep 10 19:29:00.899422 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:00.898985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79d67c4d47-5928k" event={"ID":"f7507dba-9d78-49e1-a58a-2662b355f070","Type":"ContainerStarted","Data":"bb8a206ecdd1b38eeceb59204494a0ae6fe16879b53c39f4e54996192083563e"}
Sep 10 19:29:00.916803 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:00.916748    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79d67c4d47-5928k" podStartSLOduration=1.2826674900000001 podStartE2EDuration="1.916730233s" totalImagesPullingTime="634.062743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:00.913999216 +0000 UTC m=+799.735055350" watchObservedRunningTime="2026-09-10 19:29:00.916730233 +0000 UTC m=+799.737786365"
Sep 10 19:29:00.941431 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:00.941386    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:29:00.941767 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:00.941608    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" podUID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" containerName="authorino" containerID="cri-o://c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d" gracePeriod=30
Sep 10 19:29:01.205862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.205832    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:29:01.345740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.345696    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6drhc\" (UniqueName: \"kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc\") pod \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") "
Sep 10 19:29:01.345965 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.345768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert\") pod \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\" (UID: \"5d98edcb-003f-4524-b6d5-70c9b4b325c8\") "
Sep 10 19:29:01.348082 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.348045    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc" (OuterVolumeSpecName: "kube-api-access-6drhc") pod "5d98edcb-003f-4524-b6d5-70c9b4b325c8" (UID: "5d98edcb-003f-4524-b6d5-70c9b4b325c8"). InnerVolumeSpecName "kube-api-access-6drhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:01.356922 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.356885    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5d98edcb-003f-4524-b6d5-70c9b4b325c8" (UID: "5d98edcb-003f-4524-b6d5-70c9b4b325c8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:29:01.447521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.447436    2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5d98edcb-003f-4524-b6d5-70c9b4b325c8-tls-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:29:01.447521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.447467    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6drhc\" (UniqueName: \"kubernetes.io/projected/5d98edcb-003f-4524-b6d5-70c9b4b325c8-kube-api-access-6drhc\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:29:01.905826 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.905787    2567 generic.go:356] "Generic (PLEG): container finished" podID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" containerID="c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d" exitCode=0
Sep 10 19:29:01.905826 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.905827    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758dd97fdb-mhmx8"
Sep 10 19:29:01.906382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.905866    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" event={"ID":"5d98edcb-003f-4524-b6d5-70c9b4b325c8","Type":"ContainerDied","Data":"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"}
Sep 10 19:29:01.906382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.905904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758dd97fdb-mhmx8" event={"ID":"5d98edcb-003f-4524-b6d5-70c9b4b325c8","Type":"ContainerDied","Data":"40f0f21e8eac40ef54af9a4caa52c3230628502071cab5a801a9eeba07101744"}
Sep 10 19:29:01.906382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.905925    2567 scope.go:117] "RemoveContainer" containerID="c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"
Sep 10 19:29:01.915729 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.915710    2567 scope.go:117] "RemoveContainer" containerID="c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"
Sep 10 19:29:01.916066 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:01.916045    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d\": container with ID starting with c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d not found: ID does not exist" containerID="c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"
Sep 10 19:29:01.916132 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.916075    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d"} err="failed to get container status \"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d\": rpc error: code = NotFound desc = could not find container \"c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d\": container with ID starting with c7801b3db55eeefc1db66c9dec60cb1c2e557a3ee4b7580b191a84227433825d not found: ID does not exist"
Sep 10 19:29:01.929369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.929343    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:29:01.934792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:01.934750    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-758dd97fdb-mhmx8"]
Sep 10 19:29:03.704518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:03.704477    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" path="/var/lib/kubelet/pods/5d98edcb-003f-4524-b6d5-70c9b4b325c8/volumes"
Sep 10 19:29:20.494619 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.494586    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8072-2d6bb3"]
Sep 10 19:29:20.495043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.495008    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" containerName="authorino"
Sep 10 19:29:20.495043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.495025    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" containerName="authorino"
Sep 10 19:29:20.495136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.495096    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d98edcb-003f-4524-b6d5-70c9b4b325c8" containerName="authorino"
Sep 10 19:29:20.498366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.498349    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:20.500668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.500644    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bglfh\""
Sep 10 19:29:20.502780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.502758    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8072-2d6bb3"]
Sep 10 19:29:20.630860 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.630824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86cb8\" (UniqueName: \"kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8\") pod \"test-curl-8072-2d6bb3\" (UID: \"e85e6f77-a06a-435c-a8ca-2b906340998a\") " pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:20.731600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.731561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86cb8\" (UniqueName: \"kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8\") pod \"test-curl-8072-2d6bb3\" (UID: \"e85e6f77-a06a-435c-a8ca-2b906340998a\") " pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:20.742505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.742470    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86cb8\" (UniqueName: \"kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8\") pod \"test-curl-8072-2d6bb3\" (UID: \"e85e6f77-a06a-435c-a8ca-2b906340998a\") " pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:20.808750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:20.808664    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:21.080230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:21.080208    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8072-2d6bb3"]
Sep 10 19:29:21.082416 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:21.082392    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode85e6f77_a06a_435c_a8ca_2b906340998a.slice/crio-05fe5ce772c3475e54cbd6e5009ba8c215d65b652a495e15d8a5fa845110e5b3 WatchSource:0}: Error finding container 05fe5ce772c3475e54cbd6e5009ba8c215d65b652a495e15d8a5fa845110e5b3: Status 404 returned error can't find the container with id 05fe5ce772c3475e54cbd6e5009ba8c215d65b652a495e15d8a5fa845110e5b3
Sep 10 19:29:22.018949 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:22.018896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8072-2d6bb3" event={"ID":"e85e6f77-a06a-435c-a8ca-2b906340998a","Type":"ContainerStarted","Data":"05fe5ce772c3475e54cbd6e5009ba8c215d65b652a495e15d8a5fa845110e5b3"}
Sep 10 19:29:22.123682 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:22.123646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:29:23.025895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.025861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8072-2d6bb3" event={"ID":"e85e6f77-a06a-435c-a8ca-2b906340998a","Type":"ContainerStarted","Data":"73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20"}
Sep 10 19:29:23.044235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.044176    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8072-2d6bb3" podStartSLOduration=1.192873536 podStartE2EDuration="3.044159244s" totalImagesPullingTime="1.851285708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:23.043443302 +0000 UTC m=+821.864499435" watchObservedRunningTime="2026-09-10 19:29:23.044159244 +0000 UTC m=+821.865215376"
Sep 10 19:29:23.647795 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.647758    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8072-2d6bb3"]
Sep 10 19:29:23.652199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.652165    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8072-2d6bb3"]
Sep 10 19:29:23.702223 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.702192    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:29:23.702648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.702630    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85e6f77-a06a-435c-a8ca-2b906340998a" containerName="test-curl-8072-2d6bb3"
Sep 10 19:29:23.702739 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.702651    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85e6f77-a06a-435c-a8ca-2b906340998a" containerName="test-curl-8072-2d6bb3"
Sep 10 19:29:23.702794 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.702752    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85e6f77-a06a-435c-a8ca-2b906340998a" containerName="test-curl-8072-2d6bb3"
Sep 10 19:29:23.706006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.705985    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.708123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.708104    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-173596-dockercfg-85hnf\""
Sep 10 19:29:23.708123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.708115    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-173596\""
Sep 10 19:29:23.725183 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.725147    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:29:23.770791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.770759    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8072-b5297b"]
Sep 10 19:29:23.774391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.774373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:23.780877 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.780853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8072-b5297b"]
Sep 10 19:29:23.861420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.861387    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.861656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.861630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6n9r\" (UniqueName: \"kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.963288 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.963204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.963288 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.963267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g\") pod \"test-curl-8072-b5297b\" (UID: \"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39\") " pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:23.963471 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.963291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6n9r\" (UniqueName: \"kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.963907 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.963884    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:23.972443 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:23.972410    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6n9r\" (UniqueName: \"kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r\") pod \"payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:24.018126 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.018094    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:24.032874 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:24.032846    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8072-2d6bb3" container="test-curl-8072-2d6bb3"
Sep 10 19:29:24.064208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.064162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g\") pod \"test-curl-8072-b5297b\" (UID: \"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39\") " pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:24.073399 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.073372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g\") pod \"test-curl-8072-b5297b\" (UID: \"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39\") " pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:24.085426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.085391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:24.294202 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.294174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:29:24.296120 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:24.296096    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7147e22_23b9_4c8b_a48d_19550da8b571.slice/crio-2fe7669c41411ac3bc8aebcd30acc758236500162045392612c415bfa76d2dd9 WatchSource:0}: Error finding container 2fe7669c41411ac3bc8aebcd30acc758236500162045392612c415bfa76d2dd9: Status 404 returned error can't find the container with id 2fe7669c41411ac3bc8aebcd30acc758236500162045392612c415bfa76d2dd9
Sep 10 19:29:24.368165 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:24.368142    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8072-b5297b"]
Sep 10 19:29:24.370728 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:24.370704    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod401fb5fe_7c49_4d10_ada1_77cd0b1b9e39.slice/crio-8279e22b04f3d7d1940f6a2ecf043edefe93751a9973c456844ac42d03554ae1 WatchSource:0}: Error finding container 8279e22b04f3d7d1940f6a2ecf043edefe93751a9973c456844ac42d03554ae1: Status 404 returned error can't find the container with id 8279e22b04f3d7d1940f6a2ecf043edefe93751a9973c456844ac42d03554ae1
Sep 10 19:29:25.041994 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.041950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8072-b5297b" event={"ID":"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39","Type":"ContainerStarted","Data":"aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b"}
Sep 10 19:29:25.042432 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.042007    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8072-b5297b" event={"ID":"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39","Type":"ContainerStarted","Data":"8279e22b04f3d7d1940f6a2ecf043edefe93751a9973c456844ac42d03554ae1"}
Sep 10 19:29:25.043513 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.043487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" event={"ID":"c7147e22-23b9-4c8b-a48d-19550da8b571","Type":"ContainerStarted","Data":"2fe7669c41411ac3bc8aebcd30acc758236500162045392612c415bfa76d2dd9"}
Sep 10 19:29:25.043675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.043646    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8072-2d6bb3" podUID="e85e6f77-a06a-435c-a8ca-2b906340998a" containerName="test-curl-8072-2d6bb3" containerID="cri-o://73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20" gracePeriod=2
Sep 10 19:29:25.060643 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.060574    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8072-b5297b" podStartSLOduration=2.060555383 podStartE2EDuration="2.060555383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:25.05758905 +0000 UTC m=+823.878645185" watchObservedRunningTime="2026-09-10 19:29:25.060555383 +0000 UTC m=+823.881611512"
Sep 10 19:29:25.692605 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.692568    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8072-b5297b"]
Sep 10 19:29:25.704744 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:25.704712    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8072-b5297b"]
Sep 10 19:29:26.050159 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:26.050136    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8072-b5297b" container="test-curl-8072-b5297b"
Sep 10 19:29:27.058340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.058295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" event={"ID":"c7147e22-23b9-4c8b-a48d-19550da8b571","Type":"ContainerStarted","Data":"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"}
Sep 10 19:29:27.058667 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.058408    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:27.058717 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.058672    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8072-b5297b" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" containerName="test-curl-8072-b5297b" containerID="cri-o://aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b" gracePeriod=2
Sep 10 19:29:27.075968 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:27.075911    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8072-b5297b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:27.076117 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.076008    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" podStartSLOduration=2.325829973 podStartE2EDuration="4.075992833s" totalImagesPullingTime="1.75016286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:27.07415514 +0000 UTC m=+825.895211264" watchObservedRunningTime="2026-09-10 19:29:27.075992833 +0000 UTC m=+825.897048965"
Sep 10 19:29:27.199786 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.199759    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:27.201804 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:27.201772    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8072-b5297b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:27.293054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.293023    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86cb8\" (UniqueName: \"kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8\") pod \"e85e6f77-a06a-435c-a8ca-2b906340998a\" (UID: \"e85e6f77-a06a-435c-a8ca-2b906340998a\") "
Sep 10 19:29:27.295284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.295254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8" (OuterVolumeSpecName: "kube-api-access-86cb8") pod "e85e6f77-a06a-435c-a8ca-2b906340998a" (UID: "e85e6f77-a06a-435c-a8ca-2b906340998a"). InnerVolumeSpecName "kube-api-access-86cb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:27.394313 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.394282    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86cb8\" (UniqueName: \"kubernetes.io/projected/e85e6f77-a06a-435c-a8ca-2b906340998a-kube-api-access-86cb8\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:29:27.703115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:27.703086    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85e6f77-a06a-435c-a8ca-2b906340998a" path="/var/lib/kubelet/pods/e85e6f77-a06a-435c-a8ca-2b906340998a/volumes"
Sep 10 19:29:28.065069 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:28.064979    2567 generic.go:356] "Generic (PLEG): container finished" podID="e85e6f77-a06a-435c-a8ca-2b906340998a" containerID="73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20" exitCode=137
Sep 10 19:29:28.065069 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:28.065024    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-2d6bb3"
Sep 10 19:29:28.065524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:28.065078    2567 scope.go:117] "RemoveContainer" containerID="73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20"
Sep 10 19:29:28.073981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:28.073961    2567 scope.go:117] "RemoveContainer" containerID="73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20"
Sep 10 19:29:28.074286 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:28.074267    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20\": container with ID starting with 73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20 not found: ID does not exist" containerID="73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20"
Sep 10 19:29:28.074350 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:28.074300    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20"} err="failed to get container status \"73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20\": rpc error: code = NotFound desc = could not find container \"73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20\": container with ID starting with 73f83407206748a7ed6986302849e7181fa132bdd650a6eff3e90f3e1d34ab20 not found: ID does not exist"
Sep 10 19:29:29.211025 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:29.210995    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:29.309500 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:29.309452    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g\") pod \"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39\" (UID: \"401fb5fe-7c49-4d10-ada1-77cd0b1b9e39\") "
Sep 10 19:29:29.311565 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:29.311493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g" (OuterVolumeSpecName: "kube-api-access-hvf8g") pod "401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" (UID: "401fb5fe-7c49-4d10-ada1-77cd0b1b9e39"). InnerVolumeSpecName "kube-api-access-hvf8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:29:29.410920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:29.410882    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvf8g\" (UniqueName: \"kubernetes.io/projected/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39-kube-api-access-hvf8g\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:29:29.702634 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:29.702599    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" path="/var/lib/kubelet/pods/401fb5fe-7c49-4d10-ada1-77cd0b1b9e39/volumes"
Sep 10 19:29:30.081569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:30.081474    2567 generic.go:356] "Generic (PLEG): container finished" podID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" containerID="aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b" exitCode=137
Sep 10 19:29:30.081569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:30.081517    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8072-b5297b"
Sep 10 19:29:30.081795 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:30.081577    2567 scope.go:117] "RemoveContainer" containerID="aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b"
Sep 10 19:29:30.090339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:30.090320    2567 scope.go:117] "RemoveContainer" containerID="aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b"
Sep 10 19:29:30.090621 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:29:30.090603    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b\": container with ID starting with aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b not found: ID does not exist" containerID="aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b"
Sep 10 19:29:30.090661 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:30.090634    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b"} err="failed to get container status \"aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b\": rpc error: code = NotFound desc = could not find container \"aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b\": container with ID starting with aeb32bc15c22db9b472c8b5c35894f0a891ffb3dfa8d6a6c29f7953022e6152b not found: ID does not exist"
Sep 10 19:29:31.634640 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.634608    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:29:31.635118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.635021    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" containerName="test-curl-8072-b5297b"
Sep 10 19:29:31.635118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.635033    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" containerName="test-curl-8072-b5297b"
Sep 10 19:29:31.635118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.635096    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="401fb5fe-7c49-4d10-ada1-77cd0b1b9e39" containerName="test-curl-8072-b5297b"
Sep 10 19:29:31.639463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.639446    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.641571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.641547    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-5352fd\""
Sep 10 19:29:31.650303 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.650277    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:29:31.677323 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.677288    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:29:31.680844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.680829    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.683049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.683027    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-5352fd-dockercfg-r4mvs\""
Sep 10 19:29:31.691593 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.691570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:29:31.732404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.732363    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.732404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.732406    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56jwg\" (UniqueName: \"kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.732612 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.732551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtws\" (UniqueName: \"kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.732612 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.732602    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.833773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.833735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtws\" (UniqueName: \"kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.834015 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.833785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.834015 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.833833    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.834015 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.833876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56jwg\" (UniqueName: \"kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.834650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.834628    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.834690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.834634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.842486 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.842456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56jwg\" (UniqueName: \"kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg\") pod \"payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.842615 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.842572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtws\" (UniqueName: \"kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws\") pod \"payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") " pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:31.950447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.950413    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:31.992802 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:31.992767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:32.228359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:32.228327    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:29:32.229550 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:32.229522    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb77ae3_30ad_4d9e_aaef_6b2c360c993f.slice/crio-c266699b36df4b1848ab19fb34c50c7ecb58bc17518272f1d9f0881cc13ec4c3 WatchSource:0}: Error finding container c266699b36df4b1848ab19fb34c50c7ecb58bc17518272f1d9f0881cc13ec4c3: Status 404 returned error can't find the container with id c266699b36df4b1848ab19fb34c50c7ecb58bc17518272f1d9f0881cc13ec4c3
Sep 10 19:29:32.280319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:32.280269    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:29:32.282405 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:32.282374    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae49c3f7_4f3d_4038_a321_1c6af524540e.slice/crio-c901a57f097693a327613d8cd7cac572ec280fbd32c495246a7ca1be85f2b17c WatchSource:0}: Error finding container c901a57f097693a327613d8cd7cac572ec280fbd32c495246a7ca1be85f2b17c: Status 404 returned error can't find the container with id c901a57f097693a327613d8cd7cac572ec280fbd32c495246a7ca1be85f2b17c
Sep 10 19:29:33.101013 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.100964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" event={"ID":"ae49c3f7-4f3d-4038-a321-1c6af524540e","Type":"ContainerStarted","Data":"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"}
Sep 10 19:29:33.101492 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.101149    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:33.101492 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.101174    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" event={"ID":"ae49c3f7-4f3d-4038-a321-1c6af524540e","Type":"ContainerStarted","Data":"c901a57f097693a327613d8cd7cac572ec280fbd32c495246a7ca1be85f2b17c"}
Sep 10 19:29:33.102485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.102457    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" event={"ID":"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f","Type":"ContainerStarted","Data":"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"}
Sep 10 19:29:33.102579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.102509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" event={"ID":"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f","Type":"ContainerStarted","Data":"c266699b36df4b1848ab19fb34c50c7ecb58bc17518272f1d9f0881cc13ec4c3"}
Sep 10 19:29:33.102579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.102553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:33.122908 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.122854    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" podStartSLOduration=2.122836156 podStartE2EDuration="2.122836156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:33.119189144 +0000 UTC m=+831.940245280" watchObservedRunningTime="2026-09-10 19:29:33.122836156 +0000 UTC m=+831.943892292"
Sep 10 19:29:33.134998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:33.134950    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" podStartSLOduration=2.134919454 podStartE2EDuration="2.134919454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:33.133685051 +0000 UTC m=+831.954741225" watchObservedRunningTime="2026-09-10 19:29:33.134919454 +0000 UTC m=+831.955975588"
Sep 10 19:29:38.067440 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:38.067406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:29:38.431763 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:38.431726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:29:44.109681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:44.109648    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:29:44.110118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:44.109715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:29:47.379225 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.379185    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:29:47.384533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.384511    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.386756 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.386731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e7e4c3-openshift-default-dockercfg-smd95\""
Sep 10 19:29:47.386902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.386753    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 19:29:47.396421 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.396396    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:29:47.481014 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.480970    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481195 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481308    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gq28\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.481429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.481410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582506 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582464    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582506 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gq28\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.582915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.583192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582948    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.583192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.582880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.583192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.583148    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.583325 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.583211    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.583361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.583336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.585152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.585122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.585311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.585292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.591844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.591824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.592045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.592028    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gq28\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28\") pod \"e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") " pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.696717 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.696681    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:47.987066 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:47.987040    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:29:47.988627 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:47.988598    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0631a46a_3e07_4048_8b1f_639d0e0a958c.slice/crio-dd9f915dc425dfe4a5d7b1af8e0e246c65a24f9c93cc17858caf89fb3cbacd10 WatchSource:0}: Error finding container dd9f915dc425dfe4a5d7b1af8e0e246c65a24f9c93cc17858caf89fb3cbacd10: Status 404 returned error can't find the container with id dd9f915dc425dfe4a5d7b1af8e0e246c65a24f9c93cc17858caf89fb3cbacd10
Sep 10 19:29:48.193339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:48.193297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerStarted","Data":"dd9f915dc425dfe4a5d7b1af8e0e246c65a24f9c93cc17858caf89fb3cbacd10"}
Sep 10 19:29:50.464492 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:50.464448    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:50.464874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:50.464539    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:50.464874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:50.464587    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:29:51.219812 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:51.219779    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerStarted","Data":"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"}
Sep 10 19:29:51.242348 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:51.242299    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podStartSLOduration=1.769036545 podStartE2EDuration="4.242280891s" totalImagesPullingTime="2.473244346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:51.240638893 +0000 UTC m=+850.061695049" watchObservedRunningTime="2026-09-10 19:29:51.242280891 +0000 UTC m=+850.063337025"
Sep 10 19:29:51.703921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:51.703889    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:51.704293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:51.703997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:52.225208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:52.225167    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:52.226232 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:52.226212    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:29:55.732564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.732529    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"]
Sep 10 19:29:55.736462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.736443    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:29:55.738817 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.738792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:29:55.738995 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.738820    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:29:55.744998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.744969    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"]
Sep 10 19:29:55.864181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.864143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh\" (UID: \"e1ede239-20f6-41cc-a78e-34a11577a642\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:29:55.965720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.965680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh\" (UID: \"e1ede239-20f6-41cc-a78e-34a11577a642\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:29:55.966518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:55.966480    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh\" (UID: \"e1ede239-20f6-41cc-a78e-34a11577a642\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:29:56.050464 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.050374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:29:56.331415 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.331332    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"]
Sep 10 19:29:56.333909 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:56.333881    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ede239_20f6_41cc_a78e_34a11577a642.slice/crio-cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529 WatchSource:0}: Error finding container cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529: Status 404 returned error can't find the container with id cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529
Sep 10 19:29:56.664187 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.664155    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:29:56.672796 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.672775    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.675069 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.675046    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e7e4c3-dockercfg-vwmm5\""
Sep 10 19:29:56.675069 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.675060    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e7e4c3\""
Sep 10 19:29:56.681028 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.681002    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:29:56.773057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.773021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.773457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.773074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k6l7\" (UniqueName: \"kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.873663 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.873632    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.873830 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.873681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k6l7\" (UniqueName: \"kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.874387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.874369    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.885216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.885191    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k6l7\" (UniqueName: \"kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7\") pod \"payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:56.985867 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:56.985784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:57.256517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:57.256433    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh" event={"ID":"e1ede239-20f6-41cc-a78e-34a11577a642","Type":"ContainerStarted","Data":"cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529"}
Sep 10 19:29:57.263420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:57.263387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:29:57.266316 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:29:57.266284    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d41a67_f096_413b_8a4b_facefc6f2ea8.slice/crio-500040782e2d3abc5583262be5bf489ffb89dfb58c312a8d9502664251c5f4e6 WatchSource:0}: Error finding container 500040782e2d3abc5583262be5bf489ffb89dfb58c312a8d9502664251c5f4e6: Status 404 returned error can't find the container with id 500040782e2d3abc5583262be5bf489ffb89dfb58c312a8d9502664251c5f4e6
Sep 10 19:29:58.265287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:58.265253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" event={"ID":"73d41a67-f096-413b-8a4b-facefc6f2ea8","Type":"ContainerStarted","Data":"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"}
Sep 10 19:29:58.265696 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:58.265308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" event={"ID":"73d41a67-f096-413b-8a4b-facefc6f2ea8","Type":"ContainerStarted","Data":"500040782e2d3abc5583262be5bf489ffb89dfb58c312a8d9502664251c5f4e6"}
Sep 10 19:29:58.265696 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:58.265467    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:29:58.283244 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:58.283189    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" podStartSLOduration=2.28316921 podStartE2EDuration="2.28316921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:58.281465993 +0000 UTC m=+857.102522128" watchObservedRunningTime="2026-09-10 19:29:58.28316921 +0000 UTC m=+857.104225341"
Sep 10 19:29:59.125614 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:59.125533    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:29:59.274485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:59.274441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh" event={"ID":"e1ede239-20f6-41cc-a78e-34a11577a642","Type":"ContainerStarted","Data":"476a700843983513abb26eccc44def399ad10121ac0729cb42834c495bf6b6ea"}
Sep 10 19:29:59.293143 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:29:59.293081    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh" podStartSLOduration=1.84937568 podStartE2EDuration="4.29306556s" totalImagesPullingTime="2.44368988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:29:59.290683208 +0000 UTC m=+858.111739344" watchObservedRunningTime="2026-09-10 19:29:59.29306556 +0000 UTC m=+858.114121693"
Sep 10 19:30:00.148447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.148417    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"]
Sep 10 19:30:00.152002 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.151972    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:00.152180 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.152149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"
Sep 10 19:30:00.157614 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.156461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"
Sep 10 19:30:00.161667 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.161641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"]
Sep 10 19:30:00.166269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.166243    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:00.169056 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.169036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"
Sep 10 19:30:00.179100 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.179078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"
Sep 10 19:30:00.287201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.286603    2567 generic.go:356] "Generic (PLEG): container finished" podID="e1ede239-20f6-41cc-a78e-34a11577a642" containerID="476a700843983513abb26eccc44def399ad10121ac0729cb42834c495bf6b6ea" exitCode=0
Sep 10 19:30:00.287201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.286702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh" event={"ID":"e1ede239-20f6-41cc-a78e-34a11577a642","Type":"ContainerDied","Data":"476a700843983513abb26eccc44def399ad10121ac0729cb42834c495bf6b6ea"}
Sep 10 19:30:00.467156 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.467120    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"]
Sep 10 19:30:00.469497 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:30:00.469462    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e1628e_4500_49b3_af1e_d1aaca494462.slice/crio-d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2 WatchSource:0}: Error finding container d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2: Status 404 returned error can't find the container with id d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2
Sep 10 19:30:00.469755 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.469733    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:00.470072 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:30:00.470052    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d67f90e_6c85_4b82_9957_b7629e4d1357.slice/crio-346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f WatchSource:0}: Error finding container 346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f: Status 404 returned error can't find the container with id 346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f
Sep 10 19:30:00.807347 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.807311    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:30:00.811079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.811060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.813723 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.813703    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-c98a20d3-kserve-self-signed-certs\""
Sep 10 19:30:00.819370 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.819343    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.848548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.848610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.849010    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7x26\" (UniqueName: \"kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.849053    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.849133    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.850037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.849192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.949866 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.949825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7x26\" (UniqueName: \"kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.949866 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.949867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.949901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.949927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.949999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950161 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.950027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.950362    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.950400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.950501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.950432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.952345 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.952324    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.952607 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.952589    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:00.959240 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:00.959221    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7x26\" (UniqueName: \"kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26\") pod \"e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") " pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:01.152457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.152418    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:01.293809 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.293768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw" event={"ID":"74e1628e-4500-49b3-af1e-d1aaca494462","Type":"ContainerStarted","Data":"d4f70187a8b94c727ff069eeb23bb35d514883c0ebbe10bf7077d24b32e03323"}
Sep 10 19:30:01.294293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.293831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw" event={"ID":"74e1628e-4500-49b3-af1e-d1aaca494462","Type":"ContainerStarted","Data":"d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2"}
Sep 10 19:30:01.295444 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.295421    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerStarted","Data":"dc42cb162b772782f0e983263c361364c81590acde76692eb35e8903442b0372"}
Sep 10 19:30:01.295512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.295468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerStarted","Data":"346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f"}
Sep 10 19:30:01.311863 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.311805    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw" podStartSLOduration=1.311789739 podStartE2EDuration="1.311789739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:01.310169405 +0000 UTC m=+860.131225539" watchObservedRunningTime="2026-09-10 19:30:01.311789739 +0000 UTC m=+860.132845871"
Sep 10 19:30:01.337960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.332079    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" podStartSLOduration=1.332057176 podStartE2EDuration="1.332057176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:01.329674039 +0000 UTC m=+860.150730173" watchObservedRunningTime="2026-09-10 19:30:01.332057176 +0000 UTC m=+860.153113310"
Sep 10 19:30:01.451439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.451241    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:30:01.478259 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.478236    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:30:01.555309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.555277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca\") pod \"e1ede239-20f6-41cc-a78e-34a11577a642\" (UID: \"e1ede239-20f6-41cc-a78e-34a11577a642\") "
Sep 10 19:30:01.555592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.555572    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1ede239-20f6-41cc-a78e-34a11577a642" (UID: "e1ede239-20f6-41cc-a78e-34a11577a642"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:01.555659 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:01.555648    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1ede239-20f6-41cc-a78e-34a11577a642-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:30:02.302460 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.302427    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"
Sep 10 19:30:02.302906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.302436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh" event={"ID":"e1ede239-20f6-41cc-a78e-34a11577a642","Type":"ContainerDied","Data":"cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529"}
Sep 10 19:30:02.302906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.302589    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cce32d3faa87baab53c27bee6c290802aa27ec151168ce63cea5d667843c6529"
Sep 10 19:30:02.304147 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.304118    2567 generic.go:356] "Generic (PLEG): container finished" podID="74e1628e-4500-49b3-af1e-d1aaca494462" containerID="d4f70187a8b94c727ff069eeb23bb35d514883c0ebbe10bf7077d24b32e03323" exitCode=0
Sep 10 19:30:02.304289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.304212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw" event={"ID":"74e1628e-4500-49b3-af1e-d1aaca494462","Type":"ContainerDied","Data":"d4f70187a8b94c727ff069eeb23bb35d514883c0ebbe10bf7077d24b32e03323"}
Sep 10 19:30:02.305982 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.305959    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" event={"ID":"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2","Type":"ContainerStarted","Data":"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"}
Sep 10 19:30:02.306107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.305987    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" event={"ID":"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2","Type":"ContainerStarted","Data":"8fb5ef124a91c0d12d4c38c6aa8bf4a28d3c4551ebcd3b28a8b0f7b6564b4639"}
Sep 10 19:30:02.307439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.307419    2567 generic.go:356] "Generic (PLEG): container finished" podID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerID="dc42cb162b772782f0e983263c361364c81590acde76692eb35e8903442b0372" exitCode=7
Sep 10 19:30:02.307550 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.307476    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerDied","Data":"dc42cb162b772782f0e983263c361364c81590acde76692eb35e8903442b0372"}
Sep 10 19:30:02.307594 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.307583    2567 scope.go:117] "RemoveContainer" containerID="dc42cb162b772782f0e983263c361364c81590acde76692eb35e8903442b0372"
Sep 10 19:30:02.466644 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:02.466531    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" podStartSLOduration=2.466511271 podStartE2EDuration="2.466511271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:02.462707405 +0000 UTC m=+861.283763538" watchObservedRunningTime="2026-09-10 19:30:02.466511271 +0000 UTC m=+861.287567407"
Sep 10 19:30:03.316179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:03.316142    2567 generic.go:356] "Generic (PLEG): container finished" podID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerID="f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca" exitCode=7
Sep 10 19:30:03.316687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:03.316234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerDied","Data":"f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca"}
Sep 10 19:30:03.316687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:03.316291    2567 scope.go:117] "RemoveContainer" containerID="dc42cb162b772782f0e983263c361364c81590acde76692eb35e8903442b0372"
Sep 10 19:30:03.316687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:03.316440    2567 scope.go:117] "RemoveContainer" containerID="f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca"
Sep 10 19:30:03.316844 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:30:03.316744    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq_odh-ai-gateway-infra(4d67f90e-6c85-4b82-9957-b7629e4d1357)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357"
Sep 10 19:30:03.505543 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:03.505478    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"
Sep 10 19:30:04.328914 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:04.328876    2567 scope.go:117] "RemoveContainer" containerID="f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca"
Sep 10 19:30:04.329444 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:30:04.329213    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq_odh-ai-gateway-infra(4d67f90e-6c85-4b82-9957-b7629e4d1357)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357"
Sep 10 19:30:04.331806 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:04.331774    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw" event={"ID":"74e1628e-4500-49b3-af1e-d1aaca494462","Type":"ContainerDied","Data":"d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2"}
Sep 10 19:30:04.331991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:04.331832    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d844d545d3155cff88b9c86da9a5fd81ccc54d80ad3200a68f630a9350b089f2"
Sep 10 19:30:04.331991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:04.331862    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"
Sep 10 19:30:08.268102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:08.268059    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 19:30:08.268572 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:08.268146    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 19:30:09.276998 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.276964    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:30:09.374080 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.374001    2567 generic.go:356] "Generic (PLEG): container finished" podID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerID="0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee" exitCode=0
Sep 10 19:30:09.374220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.374076    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerDied","Data":"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"}
Sep 10 19:30:09.374495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.374481    2567 scope.go:117] "RemoveContainer" containerID="0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"
Sep 10 19:30:09.376611 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.376579    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:09.376706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.376647    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:09.376706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:09.376680    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 19:30:10.383034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:10.382993    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerStarted","Data":"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"}
Sep 10 19:30:10.697925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:10.697883    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:30:10.698226 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:10.698202    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:10.698320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:10.698261    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:11.152826 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.152787    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:11.153060 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.152844    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:11.165729 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.165705    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:11.399711 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.399682    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:30:11.697310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.697274    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:11.697536 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:11.697348    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:12.473839 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:12.473803    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"]
Sep 10 19:30:12.476108 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:12.476088    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7fa2c81-nvdfh"]
Sep 10 19:30:12.697732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:12.697691    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:12.697910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:12.697783    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:13.697241 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:13.697208    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:13.697634 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:13.697273    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:13.703137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:13.703107    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1ede239-20f6-41cc-a78e-34a11577a642" path="/var/lib/kubelet/pods/e1ede239-20f6-41cc-a78e-34a11577a642/volumes"
Sep 10 19:30:14.697210 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:14.697170    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:14.697395 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:14.697253    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:15.633495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:15.633442    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:30:15.696164 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:15.696117    2567 scope.go:117] "RemoveContainer" containerID="f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca"
Sep 10 19:30:15.697278 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:15.697175    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 19:30:15.697278 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:15.697232    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 19:30:16.420216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:16.420182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerStarted","Data":"741aeb418315cb73bf970d794003e816af792084b7616055f56522a5c2eb4ea1"}
Sep 10 19:30:16.700542 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:16.700464    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:30:16.700780 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:16.700759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:30:16.701476 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:16.701462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:30:17.429264 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:17.429229    2567 generic.go:356] "Generic (PLEG): container finished" podID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerID="741aeb418315cb73bf970d794003e816af792084b7616055f56522a5c2eb4ea1" exitCode=0
Sep 10 19:30:17.429721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:17.429311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerDied","Data":"741aeb418315cb73bf970d794003e816af792084b7616055f56522a5c2eb4ea1"}
Sep 10 19:30:17.429721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:17.429380    2567 scope.go:117] "RemoveContainer" containerID="f83b82d274cb5eb05e476669fb7cf86a2bc55c4124ca4902ec9c89bf8b4d1aca"
Sep 10 19:30:17.456286 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:17.456255    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:18.585171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:18.585140    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"
Sep 10 19:30:19.445231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.445200    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"
Sep 10 19:30:19.445411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.445232    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq" event={"ID":"4d67f90e-6c85-4b82-9957-b7629e4d1357","Type":"ContainerDied","Data":"346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f"}
Sep 10 19:30:19.445411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.445278    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="346a00cde5f0d495faa61849d5394884d4e786ffd96204858d62bc8399bfee1f"
Sep 10 19:30:19.485361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.485323    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:19.493199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.493170    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-e7e4c3-29817810-lf8lq"]
Sep 10 19:30:19.705678 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:19.705594    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" path="/var/lib/kubelet/pods/4d67f90e-6c85-4b82-9957-b7629e4d1357/volumes"
Sep 10 19:30:22.103359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103316    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:30:22.103806 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103793    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103808    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103818    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74e1628e-4500-49b3-af1e-d1aaca494462" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103824    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="74e1628e-4500-49b3-af1e-d1aaca494462" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103829    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103836    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103843    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.103851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103848    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103876    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1ede239-20f6-41cc-a78e-34a11577a642" containerName="revoke-keys"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103886    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1ede239-20f6-41cc-a78e-34a11577a642" containerName="revoke-keys"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103978    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1ede239-20f6-41cc-a78e-34a11577a642" containerName="revoke-keys"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103986    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.103993    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="74e1628e-4500-49b3-af1e-d1aaca494462" containerName="cleanup"
Sep 10 19:30:22.104084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.104003    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:22.107265 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.107249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.110391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.110370    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-5352fd\"/\"default-dockercfg-6bvjt\""
Sep 10 19:30:22.110526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.110401    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-5352fd\"/\"kube-root-ca.crt\""
Sep 10 19:30:22.110526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.110468    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-5352fd\"/\"openshift-service-ca.crt\""
Sep 10 19:30:22.110526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.110489    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-5352fd\"/\"autores-model-5352fd-kserve-self-signed-certs\""
Sep 10 19:30:22.115731 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.115713    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:30:22.174142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.174349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgsbg\" (UniqueName: \"kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.174349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.174349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.174349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.174536 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.174390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275175 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275238    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275363 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgsbg\" (UniqueName: \"kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275638 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.275692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.275669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.277613 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.277584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.277864 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.277846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.284300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.284275    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgsbg\" (UniqueName: \"kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg\") pod \"autores-model-5352fd-kserve-8dd9f647-sjdpc\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") " pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.417380 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.417343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:22.693688 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:22.693656    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:30:22.695374 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:30:22.695344    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c8bd0c3_3d3e_4749_b75c_2fced75daa6d.slice/crio-1d0ab379f6ab33169e667e19f66f7ee2247cb8900d030ac8346e36a2a8fd5505 WatchSource:0}: Error finding container 1d0ab379f6ab33169e667e19f66f7ee2247cb8900d030ac8346e36a2a8fd5505: Status 404 returned error can't find the container with id 1d0ab379f6ab33169e667e19f66f7ee2247cb8900d030ac8346e36a2a8fd5505
Sep 10 19:30:23.471736 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:23.471698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" event={"ID":"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d","Type":"ContainerStarted","Data":"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"}
Sep 10 19:30:23.472169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:23.471745    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" event={"ID":"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d","Type":"ContainerStarted","Data":"1d0ab379f6ab33169e667e19f66f7ee2247cb8900d030ac8346e36a2a8fd5505"}
Sep 10 19:30:23.490224 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:23.490177    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" podStartSLOduration=1.490160846 podStartE2EDuration="1.490160846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:23.487147829 +0000 UTC m=+882.308203961" watchObservedRunningTime="2026-09-10 19:30:23.490160846 +0000 UTC m=+882.311216978"
Sep 10 19:30:32.417893 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:32.417857    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:32.418377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:32.417959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:32.430455 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:32.430432    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:32.542928 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:32.542898    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:30:39.640519 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:39.640441    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:30:41.641538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.641507    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"]
Sep 10 19:30:41.642046 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.642030    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d67f90e-6c85-4b82-9957-b7629e4d1357" containerName="cleanup"
Sep 10 19:30:41.645356 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.645339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:41.647840 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.647818    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:30:41.647977 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.647840    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:30:41.652871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.652840    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"]
Sep 10 19:30:41.663778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.663748    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln\" (UID: \"07077c6b-1c44-49fb-af17-5de1fb6ad2cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:41.692878 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.692847    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:30:41.694372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.694348    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:30:41.701220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.701197    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:30:41.702541 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.702522    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:30:41.764482 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.764441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln\" (UID: \"07077c6b-1c44-49fb-af17-5de1fb6ad2cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:41.765068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.765049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln\" (UID: \"07077c6b-1c44-49fb-af17-5de1fb6ad2cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:41.958302 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:41.958274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:42.235055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:42.235031    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"]
Sep 10 19:30:42.236804 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:30:42.236781    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07077c6b_1c44_49fb_af17_5de1fb6ad2cb.slice/crio-0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660 WatchSource:0}: Error finding container 0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660: Status 404 returned error can't find the container with id 0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660
Sep 10 19:30:42.596068 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:42.596034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln" event={"ID":"07077c6b-1c44-49fb-af17-5de1fb6ad2cb","Type":"ContainerStarted","Data":"1c0fd13c225dbcb08cea0b5bc041184c5d30baa0b147c2f38a7f94bdfaa5e6ce"}
Sep 10 19:30:42.596263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:42.596093    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln" event={"ID":"07077c6b-1c44-49fb-af17-5de1fb6ad2cb","Type":"ContainerStarted","Data":"0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660"}
Sep 10 19:30:42.611973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:42.611899    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln" podStartSLOduration=1.611883006 podStartE2EDuration="1.611883006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:30:42.61114817 +0000 UTC m=+901.432204307" watchObservedRunningTime="2026-09-10 19:30:42.611883006 +0000 UTC m=+901.432939138"
Sep 10 19:30:43.603382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:43.603348    2567 generic.go:356] "Generic (PLEG): container finished" podID="07077c6b-1c44-49fb-af17-5de1fb6ad2cb" containerID="1c0fd13c225dbcb08cea0b5bc041184c5d30baa0b147c2f38a7f94bdfaa5e6ce" exitCode=0
Sep 10 19:30:43.603773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:43.603429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln" event={"ID":"07077c6b-1c44-49fb-af17-5de1fb6ad2cb","Type":"ContainerDied","Data":"1c0fd13c225dbcb08cea0b5bc041184c5d30baa0b147c2f38a7f94bdfaa5e6ce"}
Sep 10 19:30:44.754142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:44.754117    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:44.800165 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:44.800133    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca\") pod \"07077c6b-1c44-49fb-af17-5de1fb6ad2cb\" (UID: \"07077c6b-1c44-49fb-af17-5de1fb6ad2cb\") "
Sep 10 19:30:44.800589 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:44.800558    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "07077c6b-1c44-49fb-af17-5de1fb6ad2cb" (UID: "07077c6b-1c44-49fb-af17-5de1fb6ad2cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:30:44.901558 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:44.901525    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/07077c6b-1c44-49fb-af17-5de1fb6ad2cb-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:30:45.616166 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:45.616135    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"
Sep 10 19:30:45.616380 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:45.616178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln" event={"ID":"07077c6b-1c44-49fb-af17-5de1fb6ad2cb","Type":"ContainerDied","Data":"0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660"}
Sep 10 19:30:45.616380 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:45.616254    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cc7ccf5185a8b9357f87c3e9d1c65d7022f48df2303471f1dda6d21bd68d660"
Sep 10 19:30:47.759213 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:47.759176    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"]
Sep 10 19:30:47.763454 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:47.763417    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ac5183cd-hshln"]
Sep 10 19:30:49.704069 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:49.704034    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07077c6b-1c44-49fb-af17-5de1fb6ad2cb" path="/var/lib/kubelet/pods/07077c6b-1c44-49fb-af17-5de1fb6ad2cb/volumes"
Sep 10 19:30:57.151924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.151879    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"]
Sep 10 19:30:57.152387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.152372    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07077c6b-1c44-49fb-af17-5de1fb6ad2cb" containerName="revoke-keys"
Sep 10 19:30:57.152387 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.152388    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="07077c6b-1c44-49fb-af17-5de1fb6ad2cb" containerName="revoke-keys"
Sep 10 19:30:57.152496 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.152458    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="07077c6b-1c44-49fb-af17-5de1fb6ad2cb" containerName="revoke-keys"
Sep 10 19:30:57.155871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.155856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:57.158035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.158006    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:30:57.158181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.158048    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:30:57.162988 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.162960    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"]
Sep 10 19:30:57.220898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.220861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-5352fd-kg67k\" (UID: \"e5d9fdd8-8438-4fb0-8930-e12470afe04b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:57.321923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.321894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-5352fd-kg67k\" (UID: \"e5d9fdd8-8438-4fb0-8930-e12470afe04b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:57.322517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.322497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-5352fd-kg67k\" (UID: \"e5d9fdd8-8438-4fb0-8930-e12470afe04b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:57.467999 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.467892    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:57.740749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:57.740717    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"]
Sep 10 19:30:57.742475 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:30:57.742444    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d9fdd8_8438_4fb0_8930_e12470afe04b.slice/crio-244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f WatchSource:0}: Error finding container 244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f: Status 404 returned error can't find the container with id 244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f
Sep 10 19:30:58.695145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:58.695062    2567 generic.go:356] "Generic (PLEG): container finished" podID="e5d9fdd8-8438-4fb0-8930-e12470afe04b" containerID="c75edff4073773dcc62f82f1357de49756c9030658f80691ce0ffc417952e430" exitCode=0
Sep 10 19:30:58.695145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:58.695099    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k" event={"ID":"e5d9fdd8-8438-4fb0-8930-e12470afe04b","Type":"ContainerDied","Data":"c75edff4073773dcc62f82f1357de49756c9030658f80691ce0ffc417952e430"}
Sep 10 19:30:58.695145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:58.695135    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k" event={"ID":"e5d9fdd8-8438-4fb0-8930-e12470afe04b","Type":"ContainerStarted","Data":"244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f"}
Sep 10 19:30:59.842546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:59.842524    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:30:59.948481 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:59.948449    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca\") pod \"e5d9fdd8-8438-4fb0-8930-e12470afe04b\" (UID: \"e5d9fdd8-8438-4fb0-8930-e12470afe04b\") "
Sep 10 19:30:59.948832 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:30:59.948800    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5d9fdd8-8438-4fb0-8930-e12470afe04b" (UID: "e5d9fdd8-8438-4fb0-8930-e12470afe04b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:00.049863 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:00.049774    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d9fdd8-8438-4fb0-8930-e12470afe04b-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:00.436100 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:00.436062    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:31:00.709487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:00.709406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k" event={"ID":"e5d9fdd8-8438-4fb0-8930-e12470afe04b","Type":"ContainerDied","Data":"244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f"}
Sep 10 19:31:00.709487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:00.709458    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="244b0af97d437fdf2e17ffd43395700771e921baa1ace3f27f8a40489648e06f"
Sep 10 19:31:00.709487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:00.709458    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"
Sep 10 19:31:07.680647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.680610    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:31:07.681118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.680880    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" podUID="ae49c3f7-4f3d-4038-a321-1c6af524540e" containerName="payload-processing" containerID="cri-o://682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295" gracePeriod=30
Sep 10 19:31:07.687564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.686553    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:31:07.687564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.686849    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" podUID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" containerName="payload-pre-processing" containerID="cri-o://3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098" gracePeriod=30
Sep 10 19:31:07.864751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.864692    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"]
Sep 10 19:31:07.868115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.868088    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-5352fd-kg67k"]
Sep 10 19:31:07.944512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.944446    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:31:07.977119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:07.977092    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:31:08.033818 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.033783    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vtws\" (UniqueName: \"kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws\") pod \"ae49c3f7-4f3d-4038-a321-1c6af524540e\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") "
Sep 10 19:31:08.034093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.033843    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume\") pod \"ae49c3f7-4f3d-4038-a321-1c6af524540e\" (UID: \"ae49c3f7-4f3d-4038-a321-1c6af524540e\") "
Sep 10 19:31:08.034284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.034256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae49c3f7-4f3d-4038-a321-1c6af524540e" (UID: "ae49c3f7-4f3d-4038-a321-1c6af524540e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:08.036112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.036089    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws" (OuterVolumeSpecName: "kube-api-access-2vtws") pod "ae49c3f7-4f3d-4038-a321-1c6af524540e" (UID: "ae49c3f7-4f3d-4038-a321-1c6af524540e"). InnerVolumeSpecName "kube-api-access-2vtws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:08.134729 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.134697    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume\") pod \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") "
Sep 10 19:31:08.134923 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.134781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56jwg\" (UniqueName: \"kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg\") pod \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\" (UID: \"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f\") "
Sep 10 19:31:08.135115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.135090    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" (UID: "6cb77ae3-30ad-4d9e-aaef-6b2c360c993f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:08.135207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.135194    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-plugins-config-volume\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:08.135244 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.135209    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vtws\" (UniqueName: \"kubernetes.io/projected/ae49c3f7-4f3d-4038-a321-1c6af524540e-kube-api-access-2vtws\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:08.135244 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.135218    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae49c3f7-4f3d-4038-a321-1c6af524540e-plugins-config-volume\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:08.136834 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.136812    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg" (OuterVolumeSpecName: "kube-api-access-56jwg") pod "6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" (UID: "6cb77ae3-30ad-4d9e-aaef-6b2c360c993f"). InnerVolumeSpecName "kube-api-access-56jwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:08.236699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.236619    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56jwg\" (UniqueName: \"kubernetes.io/projected/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f-kube-api-access-56jwg\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:08.757334 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.757298    2567 generic.go:356] "Generic (PLEG): container finished" podID="ae49c3f7-4f3d-4038-a321-1c6af524540e" containerID="682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295" exitCode=0
Sep 10 19:31:08.757774 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.757339    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"
Sep 10 19:31:08.757774 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.757379    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" event={"ID":"ae49c3f7-4f3d-4038-a321-1c6af524540e","Type":"ContainerDied","Data":"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"}
Sep 10 19:31:08.757774 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.757425    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt" event={"ID":"ae49c3f7-4f3d-4038-a321-1c6af524540e","Type":"ContainerDied","Data":"c901a57f097693a327613d8cd7cac572ec280fbd32c495246a7ca1be85f2b17c"}
Sep 10 19:31:08.757774 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.757446    2567 scope.go:117] "RemoveContainer" containerID="682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"
Sep 10 19:31:08.759136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.759112    2567 generic.go:356] "Generic (PLEG): container finished" podID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" containerID="3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098" exitCode=0
Sep 10 19:31:08.759216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.759158    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"
Sep 10 19:31:08.759216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.759166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" event={"ID":"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f","Type":"ContainerDied","Data":"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"}
Sep 10 19:31:08.759216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.759207    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh" event={"ID":"6cb77ae3-30ad-4d9e-aaef-6b2c360c993f","Type":"ContainerDied","Data":"c266699b36df4b1848ab19fb34c50c7ecb58bc17518272f1d9f0881cc13ec4c3"}
Sep 10 19:31:08.769385 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.769369    2567 scope.go:117] "RemoveContainer" containerID="682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"
Sep 10 19:31:08.769717 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:31:08.769697    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295\": container with ID starting with 682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295 not found: ID does not exist" containerID="682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"
Sep 10 19:31:08.769798 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.769726    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295"} err="failed to get container status \"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295\": rpc error: code = NotFound desc = could not find container \"682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295\": container with ID starting with 682f7e6740cde9dc8e667c014254ec5917124a2d533cfc99d0ea0072f6a53295 not found: ID does not exist"
Sep 10 19:31:08.769798 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.769743    2567 scope.go:117] "RemoveContainer" containerID="3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"
Sep 10 19:31:08.778399 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.778381    2567 scope.go:117] "RemoveContainer" containerID="3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"
Sep 10 19:31:08.778664 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:31:08.778647    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098\": container with ID starting with 3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098 not found: ID does not exist" containerID="3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"
Sep 10 19:31:08.778710 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.778673    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098"} err="failed to get container status \"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098\": rpc error: code = NotFound desc = could not find container \"3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098\": container with ID starting with 3e103d516f5eef4d6c52402f1abf3990bd357e5232ca1e3df86a23c6e6e09098 not found: ID does not exist"
Sep 10 19:31:08.783368 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.783341    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:31:08.789922 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.789902    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-5352fd-6fc6477778-6vpmh"]
Sep 10 19:31:08.800701 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.800677    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:31:08.803118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:08.803100    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-5352fd-566d8c6b5f-fcpmt"]
Sep 10 19:31:09.702808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:09.702775    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" path="/var/lib/kubelet/pods/6cb77ae3-30ad-4d9e-aaef-6b2c360c993f/volumes"
Sep 10 19:31:09.703227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:09.703213    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae49c3f7-4f3d-4038-a321-1c6af524540e" path="/var/lib/kubelet/pods/ae49c3f7-4f3d-4038-a321-1c6af524540e/volumes"
Sep 10 19:31:09.703531 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:09.703519    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d9fdd8-8438-4fb0-8930-e12470afe04b" path="/var/lib/kubelet/pods/e5d9fdd8-8438-4fb0-8930-e12470afe04b/volumes"
Sep 10 19:31:13.206300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:13.206259    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:31:17.535664 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.535621    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"]
Sep 10 19:31:17.536136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536081    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae49c3f7-4f3d-4038-a321-1c6af524540e" containerName="payload-processing"
Sep 10 19:31:17.536136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536095    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae49c3f7-4f3d-4038-a321-1c6af524540e" containerName="payload-processing"
Sep 10 19:31:17.536136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536116    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" containerName="payload-pre-processing"
Sep 10 19:31:17.536136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536122    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" containerName="payload-pre-processing"
Sep 10 19:31:17.536136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536137    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d9fdd8-8438-4fb0-8930-e12470afe04b" containerName="revoke-keys"
Sep 10 19:31:17.536300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536144    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d9fdd8-8438-4fb0-8930-e12470afe04b" containerName="revoke-keys"
Sep 10 19:31:17.536300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536206    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d9fdd8-8438-4fb0-8930-e12470afe04b" containerName="revoke-keys"
Sep 10 19:31:17.536300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536215    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae49c3f7-4f3d-4038-a321-1c6af524540e" containerName="payload-processing"
Sep 10 19:31:17.536300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.536220    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb77ae3-30ad-4d9e-aaef-6b2c360c993f" containerName="payload-pre-processing"
Sep 10 19:31:17.540977 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.540958    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:17.543613 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.543586    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:31:17.544032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.543825    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:31:17.546530 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.546507    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"]
Sep 10 19:31:17.630110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.630072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30f2651a-swhlh\" (UID: \"a2c56a29-dd5f-46d9-8330-48369cba2046\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:17.731062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.731025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30f2651a-swhlh\" (UID: \"a2c56a29-dd5f-46d9-8330-48369cba2046\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:17.731630 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.731612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30f2651a-swhlh\" (UID: \"a2c56a29-dd5f-46d9-8330-48369cba2046\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:17.853735 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:17.853649    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:18.125394 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:18.125367    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"]
Sep 10 19:31:18.127164 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:31:18.127141    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c56a29_dd5f_46d9_8330_48369cba2046.slice/crio-cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d WatchSource:0}: Error finding container cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d: Status 404 returned error can't find the container with id cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d
Sep 10 19:31:18.824059 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:18.823965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh" event={"ID":"a2c56a29-dd5f-46d9-8330-48369cba2046","Type":"ContainerStarted","Data":"21e1e4f6ad70e30054db678ea802fde8f24bd1a740f75fb500568aca08131d95"}
Sep 10 19:31:18.824059 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:18.824028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh" event={"ID":"a2c56a29-dd5f-46d9-8330-48369cba2046","Type":"ContainerStarted","Data":"cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d"}
Sep 10 19:31:18.841219 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:18.841166    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh" podStartSLOduration=1.841145739 podStartE2EDuration="1.841145739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:18.840482226 +0000 UTC m=+937.661538383" watchObservedRunningTime="2026-09-10 19:31:18.841145739 +0000 UTC m=+937.662201873"
Sep 10 19:31:19.031698 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.031664    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:19.035314 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.035293    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.037669 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.037644    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-c98a20d3-kserve-self-signed-certs\""
Sep 10 19:31:19.046699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.046672    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:19.147557 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmzdh\" (UniqueName: \"kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.147723 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.147723 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.147723 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147706    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.147834 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147767    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.147834 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.147808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249154 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249154 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmzdh\" (UniqueName: \"kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249776 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.249900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.249796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.251585 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.251565    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.251852 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.251834    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.257959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.257922    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmzdh\" (UniqueName: \"kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh\") pod \"e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") " pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.345556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.345518    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:19.622816 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.622790    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:19.624148 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:31:19.624120    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61722767_dbe7_4831_aa2c_6f60d98a5750.slice/crio-909b77c0f7e8b1248447567ab93818fed846a01a3201e6db2cd0e5cfe1b15af9 WatchSource:0}: Error finding container 909b77c0f7e8b1248447567ab93818fed846a01a3201e6db2cd0e5cfe1b15af9: Status 404 returned error can't find the container with id 909b77c0f7e8b1248447567ab93818fed846a01a3201e6db2cd0e5cfe1b15af9
Sep 10 19:31:19.831381 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.831345    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2c56a29-dd5f-46d9-8330-48369cba2046" containerID="21e1e4f6ad70e30054db678ea802fde8f24bd1a740f75fb500568aca08131d95" exitCode=0
Sep 10 19:31:19.831810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.831422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh" event={"ID":"a2c56a29-dd5f-46d9-8330-48369cba2046","Type":"ContainerDied","Data":"21e1e4f6ad70e30054db678ea802fde8f24bd1a740f75fb500568aca08131d95"}
Sep 10 19:31:19.832980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.832958    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" event={"ID":"61722767-dbe7-4831-aa2c-6f60d98a5750","Type":"ContainerStarted","Data":"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"}
Sep 10 19:31:19.833077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.832986    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" event={"ID":"61722767-dbe7-4831-aa2c-6f60d98a5750","Type":"ContainerStarted","Data":"909b77c0f7e8b1248447567ab93818fed846a01a3201e6db2cd0e5cfe1b15af9"}
Sep 10 19:31:19.866709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:19.866661    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" podStartSLOduration=0.866643243 podStartE2EDuration="866.643243ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:19.865798854 +0000 UTC m=+938.686855066" watchObservedRunningTime="2026-09-10 19:31:19.866643243 +0000 UTC m=+938.687699378"
Sep 10 19:31:20.327092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:20.327004    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:31:20.989918 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:20.989891    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:21.063869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.063838    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca\") pod \"a2c56a29-dd5f-46d9-8330-48369cba2046\" (UID: \"a2c56a29-dd5f-46d9-8330-48369cba2046\") "
Sep 10 19:31:21.064217 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.064191    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2c56a29-dd5f-46d9-8330-48369cba2046" (UID: "a2c56a29-dd5f-46d9-8330-48369cba2046"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:31:21.164978 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.164948    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c56a29-dd5f-46d9-8330-48369cba2046-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:21.854117 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.854088    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"
Sep 10 19:31:21.854293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.854088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh" event={"ID":"a2c56a29-dd5f-46d9-8330-48369cba2046","Type":"ContainerDied","Data":"cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d"}
Sep 10 19:31:21.854293 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:21.854212    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cedfb36c6b9074cc30d8afb853dee4a4ec07f5eed9c55c39e16feee5ac38ba6d"
Sep 10 19:31:23.655683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:23.655647    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"]
Sep 10 19:31:23.658452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:23.658427    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30f2651a-swhlh"]
Sep 10 19:31:23.702470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:23.702439    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c56a29-dd5f-46d9-8330-48369cba2046" path="/var/lib/kubelet/pods/a2c56a29-dd5f-46d9-8330-48369cba2046/volumes"
Sep 10 19:31:28.226603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.226526    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" podUID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" containerName="main" containerID="cri-o://797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6" gracePeriod=45
Sep 10 19:31:28.479797 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.479736    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:31:28.537397 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537365    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgsbg\" (UniqueName: \"kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537483    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537580    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537624    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home\") pod \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\" (UID: \"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d\") "
Sep 10 19:31:28.537810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537791    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:28.537944 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537835    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:28.538022 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.537963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home" (OuterVolumeSpecName: "home") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:28.538077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.538031    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.538077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.538047    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.539918 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.539885    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg" (OuterVolumeSpecName: "kube-api-access-bgsbg") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "kube-api-access-bgsbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:28.540058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.539958    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:31:28.540058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.539952    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm" (OuterVolumeSpecName: "dshm") pod "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" (UID: "5c8bd0c3-3d3e-4749-b75c-2fced75daa6d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:28.606879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.606845    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:31:28.607331 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607315    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c56a29-dd5f-46d9-8330-48369cba2046" containerName="revoke-keys"
Sep 10 19:31:28.607375 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607334    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c56a29-dd5f-46d9-8330-48369cba2046" containerName="revoke-keys"
Sep 10 19:31:28.607375 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607344    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" containerName="main"
Sep 10 19:31:28.607375 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607350    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" containerName="main"
Sep 10 19:31:28.607521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607439    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" containerName="main"
Sep 10 19:31:28.607521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.607447    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c56a29-dd5f-46d9-8330-48369cba2046" containerName="revoke-keys"
Sep 10 19:31:28.610651 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.610637    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.612822 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.612794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-173596\"/\"default-dockercfg-p5vzc\""
Sep 10 19:31:28.612983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.612799    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-173596\"/\"test-model-173596-kserve-self-signed-certs\""
Sep 10 19:31:28.612983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.612844    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-173596\"/\"kube-root-ca.crt\""
Sep 10 19:31:28.613137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.613006    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-173596\"/\"openshift-service-ca.crt\""
Sep 10 19:31:28.618120 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.618098    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:31:28.638944 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.638908    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.639086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.638954    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.639086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.638965    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.639086 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.638974    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgsbg\" (UniqueName: \"kubernetes.io/projected/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d-kube-api-access-bgsbg\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:28.740190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740097    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.740384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.740460 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740398    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.740524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6qbx\" (UniqueName: \"kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.740582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.740582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.740567    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.841734 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qbx\" (UniqueName: \"kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.841960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.841960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.841960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.841960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.842184 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.841991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.842236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.842206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.842236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.842217    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.842349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.842246    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.844052 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.844030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.844360 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.844343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.852077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.852059    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qbx\" (UniqueName: \"kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx\") pod \"test-model-173596-kserve-f5c4444c-wz8ds\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") " pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.900582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.900543    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" containerID="797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6" exitCode=0
Sep 10 19:31:28.900773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.900588    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"
Sep 10 19:31:28.900773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.900644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" event={"ID":"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d","Type":"ContainerDied","Data":"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"}
Sep 10 19:31:28.900773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.900689    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc" event={"ID":"5c8bd0c3-3d3e-4749-b75c-2fced75daa6d","Type":"ContainerDied","Data":"1d0ab379f6ab33169e667e19f66f7ee2247cb8900d030ac8346e36a2a8fd5505"}
Sep 10 19:31:28.900773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.900710    2567 scope.go:117] "RemoveContainer" containerID="797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"
Sep 10 19:31:28.911105 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.911086    2567 scope.go:117] "RemoveContainer" containerID="797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"
Sep 10 19:31:28.911389 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:31:28.911372    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6\": container with ID starting with 797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6 not found: ID does not exist" containerID="797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"
Sep 10 19:31:28.911429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.911401    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6"} err="failed to get container status \"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6\": rpc error: code = NotFound desc = could not find container \"797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6\": container with ID starting with 797d156d5227d5ec367d5bdaa10b115253583958ffa83196de80ab61d962cae6 not found: ID does not exist"
Sep 10 19:31:28.921353 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.921332    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:28.925451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.925424    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:31:28.928393 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:28.928374    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-5352fd/autores-model-5352fd-kserve-8dd9f647-sjdpc"]
Sep 10 19:31:29.197424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.197398    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:31:29.198701 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:31:29.198677    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc994a894_12bc_4f53_9bd7_f6166adc01f3.slice/crio-c746befa31ed94e4af4a6bd13ac4d5b4b1fc3e065edc8f63dc5d763113ea96ce WatchSource:0}: Error finding container c746befa31ed94e4af4a6bd13ac4d5b4b1fc3e065edc8f63dc5d763113ea96ce: Status 404 returned error can't find the container with id c746befa31ed94e4af4a6bd13ac4d5b4b1fc3e065edc8f63dc5d763113ea96ce
Sep 10 19:31:29.346176 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.346133    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:29.346668 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.346205    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:29.358574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.358548    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:29.702900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.702864    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c8bd0c3-3d3e-4749-b75c-2fced75daa6d" path="/var/lib/kubelet/pods/5c8bd0c3-3d3e-4749-b75c-2fced75daa6d/volumes"
Sep 10 19:31:29.908952 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.908890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" event={"ID":"c994a894-12bc-4f53-9bd7-f6166adc01f3","Type":"ContainerStarted","Data":"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"}
Sep 10 19:31:29.908952 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.908955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" event={"ID":"c994a894-12bc-4f53-9bd7-f6166adc01f3","Type":"ContainerStarted","Data":"c746befa31ed94e4af4a6bd13ac4d5b4b1fc3e065edc8f63dc5d763113ea96ce"}
Sep 10 19:31:29.920592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.920567    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:29.927505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:29.927464    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" podStartSLOduration=1.927451804 podStartE2EDuration="1.927451804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:29.925318761 +0000 UTC m=+948.746374905" watchObservedRunningTime="2026-09-10 19:31:29.927451804 +0000 UTC m=+948.748507936"
Sep 10 19:31:37.761688 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:37.761648    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:31:38.922155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:38.922115    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:38.922628 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:38.922177    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:38.935914 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:38.935884    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:38.991229 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:38.991192    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:31:41.505369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:41.505333    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:55.132123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:55.132091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:31:56.525994 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.525918    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" podUID="61722767-dbe7-4831-aa2c-6f60d98a5750" containerName="main" containerID="cri-o://9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389" gracePeriod=45
Sep 10 19:31:56.772853 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.772829    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:56.816986 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.816877    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.816986 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.816961    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.817230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.816997    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmzdh\" (UniqueName: \"kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.817230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817059    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.817230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817117    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.817230 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817158    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs\") pod \"61722767-dbe7-4831-aa2c-6f60d98a5750\" (UID: \"61722767-dbe7-4831-aa2c-6f60d98a5750\") "
Sep 10 19:31:56.817457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817261    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache" (OuterVolumeSpecName: "model-cache") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.817457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817357    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home" (OuterVolumeSpecName: "home") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.817457 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817392    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.817631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817593    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.817631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817615    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.817631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.817630    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.819447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.819416    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:31:56.819621 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.819600    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm" (OuterVolumeSpecName: "dshm") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:31:56.819671 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.819618    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh" (OuterVolumeSpecName: "kube-api-access-fmzdh") pod "61722767-dbe7-4831-aa2c-6f60d98a5750" (UID: "61722767-dbe7-4831-aa2c-6f60d98a5750"). InnerVolumeSpecName "kube-api-access-fmzdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:31:56.918868 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.918831    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmzdh\" (UniqueName: \"kubernetes.io/projected/61722767-dbe7-4831-aa2c-6f60d98a5750-kube-api-access-fmzdh\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.918868 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.918858    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61722767-dbe7-4831-aa2c-6f60d98a5750-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.918868 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.918869    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61722767-dbe7-4831-aa2c-6f60d98a5750-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:31:56.943606 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.943574    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"]
Sep 10 19:31:56.943999 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.943986    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61722767-dbe7-4831-aa2c-6f60d98a5750" containerName="main"
Sep 10 19:31:56.943999 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.944001    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="61722767-dbe7-4831-aa2c-6f60d98a5750" containerName="main"
Sep 10 19:31:56.944092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.944088    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="61722767-dbe7-4831-aa2c-6f60d98a5750" containerName="main"
Sep 10 19:31:56.947299 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.947284    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:31:56.949534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.949514    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:31:56.949597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.949547    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:31:56.955128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:56.955086    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"]
Sep 10 19:31:57.020058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.020008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5\" (UID: \"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:31:57.092169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.092078    2567 generic.go:356] "Generic (PLEG): container finished" podID="61722767-dbe7-4831-aa2c-6f60d98a5750" containerID="9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389" exitCode=0
Sep 10 19:31:57.092169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.092127    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"
Sep 10 19:31:57.092377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.092170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" event={"ID":"61722767-dbe7-4831-aa2c-6f60d98a5750","Type":"ContainerDied","Data":"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"}
Sep 10 19:31:57.092377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.092216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm" event={"ID":"61722767-dbe7-4831-aa2c-6f60d98a5750","Type":"ContainerDied","Data":"909b77c0f7e8b1248447567ab93818fed846a01a3201e6db2cd0e5cfe1b15af9"}
Sep 10 19:31:57.092377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.092232    2567 scope.go:117] "RemoveContainer" containerID="9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"
Sep 10 19:31:57.103198 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.103174    2567 scope.go:117] "RemoveContainer" containerID="9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"
Sep 10 19:31:57.103521 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:31:57.103501    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389\": container with ID starting with 9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389 not found: ID does not exist" containerID="9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"
Sep 10 19:31:57.103586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.103530    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389"} err="failed to get container status \"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389\": rpc error: code = NotFound desc = could not find container \"9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389\": container with ID starting with 9ea96f2c732c007d716a613e9ad043c9d87a728d38e50b1d21eb9d5b9f3c5389 not found: ID does not exist"
Sep 10 19:31:57.119884 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.119858    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:57.121603 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.121583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5\" (UID: \"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:31:57.122179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.122161    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5\" (UID: \"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:31:57.126203 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.126175    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-c98a20d3-kserve-5dfdd64575-gmxcm"]
Sep 10 19:31:57.260436 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.260410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:31:57.532737 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.532708    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"]
Sep 10 19:31:57.534645 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:31:57.534623    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3392ca50_a66a_42bd_bdd7_e69cd6b06fc3.slice/crio-825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359 WatchSource:0}: Error finding container 825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359: Status 404 returned error can't find the container with id 825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359
Sep 10 19:31:57.702877 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:57.702842    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61722767-dbe7-4831-aa2c-6f60d98a5750" path="/var/lib/kubelet/pods/61722767-dbe7-4831-aa2c-6f60d98a5750/volumes"
Sep 10 19:31:58.101808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:58.101773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5" event={"ID":"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3","Type":"ContainerStarted","Data":"e8925c08cea6a122628ab6d7613b04a0838cef45a438f760be634f4e7840cd1c"}
Sep 10 19:31:58.102032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:58.101824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5" event={"ID":"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3","Type":"ContainerStarted","Data":"825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359"}
Sep 10 19:31:58.118546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:58.118492    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5" podStartSLOduration=2.118471785 podStartE2EDuration="2.118471785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:31:58.116203648 +0000 UTC m=+976.937259782" watchObservedRunningTime="2026-09-10 19:31:58.118471785 +0000 UTC m=+976.939527952"
Sep 10 19:31:59.110448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:59.110414    2567 generic.go:356] "Generic (PLEG): container finished" podID="3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" containerID="e8925c08cea6a122628ab6d7613b04a0838cef45a438f760be634f4e7840cd1c" exitCode=0
Sep 10 19:31:59.110861 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:31:59.110498    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5" event={"ID":"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3","Type":"ContainerDied","Data":"e8925c08cea6a122628ab6d7613b04a0838cef45a438f760be634f4e7840cd1c"}
Sep 10 19:32:00.249535 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:00.249510    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:32:00.352544 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:00.352508    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca\") pod \"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3\" (UID: \"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3\") "
Sep 10 19:32:00.352906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:00.352877    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" (UID: "3392ca50-a66a-42bd-bdd7-e69cd6b06fc3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:00.453769 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:00.453735    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:01.124349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:01.124313    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"
Sep 10 19:32:01.124524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:01.124349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5" event={"ID":"3392ca50-a66a-42bd-bdd7-e69cd6b06fc3","Type":"ContainerDied","Data":"825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359"}
Sep 10 19:32:01.124524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:01.124391    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="825afd163966601c441912a52669e8235ff8cb72cfa13f98e3331b5b835f3359"
Sep 10 19:32:04.493420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:04.493325    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"]
Sep 10 19:32:04.500192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:04.500151    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-34de9e5b-87fq5"]
Sep 10 19:32:05.702790 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:05.702753    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" path="/var/lib/kubelet/pods/3392ca50-a66a-42bd-bdd7-e69cd6b06fc3/volumes"
Sep 10 19:32:11.827750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:11.827712    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:32:18.381728 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:18.381696    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:32:22.928453 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.928421    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:22.928873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.928860    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" containerName="revoke-keys"
Sep 10 19:32:22.928873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.928874    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" containerName="revoke-keys"
Sep 10 19:32:22.928993 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.928961    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3392ca50-a66a-42bd-bdd7-e69cd6b06fc3" containerName="revoke-keys"
Sep 10 19:32:22.932158 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.932141    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:22.935273 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.935240    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9a4e54\"/\"default-dockercfg-fsmkw\""
Sep 10 19:32:22.935405 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.935375    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9a4e54\"/\"openshift-service-ca.crt\""
Sep 10 19:32:22.936150 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.935567    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-9a4e54\"/\"kube-root-ca.crt\""
Sep 10 19:32:22.936150 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.935723    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-9a4e54\"/\"explicit-model-9a4e54-kserve-self-signed-certs\""
Sep 10 19:32:22.939850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:22.939828    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:23.077779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.077779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.078045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.078045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r456z\" (UniqueName: \"kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.078045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.078045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.077920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178610 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178610 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r456z\" (UniqueName: \"kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.178862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.179007 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.178989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.179075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.179039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.179181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.179160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.181062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.181035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.181241 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.181223    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.189022 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.189003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r456z\" (UniqueName: \"kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z\") pod \"explicit-model-9a4e54-kserve-5d878cc897-pgsjg\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") " pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.243439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.243407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:23.518691 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:23.518665    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:23.520807 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:32:23.520782    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf26b8f52_3e5a_4e73_8e9e_92e1aa516de8.slice/crio-60841328160f7aeb0c507399b4bd94c43ef5fe5883057f74b8ce782aeeef5417 WatchSource:0}: Error finding container 60841328160f7aeb0c507399b4bd94c43ef5fe5883057f74b8ce782aeeef5417: Status 404 returned error can't find the container with id 60841328160f7aeb0c507399b4bd94c43ef5fe5883057f74b8ce782aeeef5417
Sep 10 19:32:24.027721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:24.027689    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:32:24.266699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:24.266659    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" event={"ID":"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8","Type":"ContainerStarted","Data":"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"}
Sep 10 19:32:24.266699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:24.266705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" event={"ID":"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8","Type":"ContainerStarted","Data":"60841328160f7aeb0c507399b4bd94c43ef5fe5883057f74b8ce782aeeef5417"}
Sep 10 19:32:24.286674 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:24.286579    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" podStartSLOduration=2.286562581 podStartE2EDuration="2.286562581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:24.284337779 +0000 UTC m=+1003.105393894" watchObservedRunningTime="2026-09-10 19:32:24.286562581 +0000 UTC m=+1003.107618955"
Sep 10 19:32:33.243873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.243837    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:33.244359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.243895    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:33.256768 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.256741    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:33.331236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.331207    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:33.401700 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.401654    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" podUID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" containerName="main" containerID="cri-o://8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be" gracePeriod=45
Sep 10 19:32:33.651735 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.651710    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:32:33.672847 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.672820    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.672884    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.672951    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.672975    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673047 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673017    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7x26\" (UniqueName: \"kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673116    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:33.673287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673149    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:33.673400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673292    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home\") pod \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\" (UID: \"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2\") "
Sep 10 19:32:33.673638 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673604    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home" (OuterVolumeSpecName: "home") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:33.673747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673671    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:33.673747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.673691    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:33.675507 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.675477    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26" (OuterVolumeSpecName: "kube-api-access-b7x26") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "kube-api-access-b7x26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:33.675608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.675545    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm" (OuterVolumeSpecName: "dshm") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:33.675801 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.675778    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" (UID: "966916a7-6b5e-4efb-8182-ba9c4c9bc9a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:33.774595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.774502    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:33.774595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.774538    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:33.774595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.774552    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:33.774595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:33.774560    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7x26\" (UniqueName: \"kubernetes.io/projected/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2-kube-api-access-b7x26\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:34.326747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.326714    2567 generic.go:356] "Generic (PLEG): container finished" podID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" containerID="8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be" exitCode=0
Sep 10 19:32:34.327296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.326763    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"
Sep 10 19:32:34.327296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.326799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" event={"ID":"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2","Type":"ContainerDied","Data":"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"}
Sep 10 19:32:34.327296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.326843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd" event={"ID":"966916a7-6b5e-4efb-8182-ba9c4c9bc9a2","Type":"ContainerDied","Data":"8fb5ef124a91c0d12d4c38c6aa8bf4a28d3c4551ebcd3b28a8b0f7b6564b4639"}
Sep 10 19:32:34.327296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.326861    2567 scope.go:117] "RemoveContainer" containerID="8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"
Sep 10 19:32:34.336927 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.336906    2567 scope.go:117] "RemoveContainer" containerID="8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"
Sep 10 19:32:34.337206 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:32:34.337187    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be\": container with ID starting with 8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be not found: ID does not exist" containerID="8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"
Sep 10 19:32:34.337269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.337215    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be"} err="failed to get container status \"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be\": rpc error: code = NotFound desc = could not find container \"8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be\": container with ID starting with 8b41784ba079906639f959c099dc3dceca16451e97dbdf6ad3b88c8322dd96be not found: ID does not exist"
Sep 10 19:32:34.346121 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.346092    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:32:34.354364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:34.354343    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-c98a20d3-kserve-57b6469545-p48kd"]
Sep 10 19:32:35.703206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:35.703176    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" path="/var/lib/kubelet/pods/966916a7-6b5e-4efb-8182-ba9c4c9bc9a2/volumes"
Sep 10 19:32:37.622808 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.622781    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:32:37.623242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.623228    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" containerName="main"
Sep 10 19:32:37.623296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.623244    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" containerName="main"
Sep 10 19:32:37.623340 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.623313    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="966916a7-6b5e-4efb-8182-ba9c4c9bc9a2" containerName="main"
Sep 10 19:32:37.627740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.627723    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.631169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.631145    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e7e4c3\"/\"test-model-e7e4c3-kserve-self-signed-certs\""
Sep 10 19:32:37.631169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.631160    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e7e4c3\"/\"openshift-service-ca.crt\""
Sep 10 19:32:37.631352 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.631199    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e7e4c3\"/\"kube-root-ca.crt\""
Sep 10 19:32:37.631352 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.631199    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e7e4c3\"/\"default-dockercfg-ktdxq\""
Sep 10 19:32:37.637513 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.637490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:32:37.713092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.713092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.713358 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713132    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.713358 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713216    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.713358 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6cpj\" (UniqueName: \"kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.713358 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.713315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814765    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6cpj\" (UniqueName: \"kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.814993 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.814799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.815058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.815006    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.815058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.815049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.815204 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.815175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.816842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.816816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.817118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.817099    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.824801 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.824777    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6cpj\" (UniqueName: \"kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj\") pod \"test-model-e7e4c3-kserve-84c785cb8c-2xb6n\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") " pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:37.940001 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:37.939965    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:38.218484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:38.218457    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:32:38.220140 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:32:38.220114    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc564ac30_6f96_4881_adaf_a168f2a92ad4.slice/crio-f21a9e5674e6906a9e2478b890478639a4fe2740fc2b0e2851f731a9f74b59c9 WatchSource:0}: Error finding container f21a9e5674e6906a9e2478b890478639a4fe2740fc2b0e2851f731a9f74b59c9: Status 404 returned error can't find the container with id f21a9e5674e6906a9e2478b890478639a4fe2740fc2b0e2851f731a9f74b59c9
Sep 10 19:32:38.355889 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:38.355855    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" event={"ID":"c564ac30-6f96-4881-adaf-a168f2a92ad4","Type":"ContainerStarted","Data":"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"}
Sep 10 19:32:38.355889 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:38.355896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" event={"ID":"c564ac30-6f96-4881-adaf-a168f2a92ad4","Type":"ContainerStarted","Data":"f21a9e5674e6906a9e2478b890478639a4fe2740fc2b0e2851f731a9f74b59c9"}
Sep 10 19:32:38.374805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:38.374756    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" podStartSLOduration=1.374739927 podStartE2EDuration="1.374739927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:38.373388332 +0000 UTC m=+1017.194444464" watchObservedRunningTime="2026-09-10 19:32:38.374739927 +0000 UTC m=+1017.195796059"
Sep 10 19:32:39.948075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:39.948041    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:40.090261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.090231    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"]
Sep 10 19:32:40.093950 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.093915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:40.096445 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.096421    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:32:40.096571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.096463    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:32:40.102763 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.102313    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"]
Sep 10 19:32:40.140191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.140159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb\" (UID: \"6a4fc587-97d0-4fad-950f-3b1500773e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:40.241794 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.241688    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb\" (UID: \"6a4fc587-97d0-4fad-950f-3b1500773e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:40.242384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.242361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb\" (UID: \"6a4fc587-97d0-4fad-950f-3b1500773e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:40.406576 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.406545    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:40.541628 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.541551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:32:40.698309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:40.698283    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"]
Sep 10 19:32:40.700399 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:32:40.700376    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4fc587_97d0_4fad_950f_3b1500773e2d.slice/crio-b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5 WatchSource:0}: Error finding container b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5: Status 404 returned error can't find the container with id b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5
Sep 10 19:32:41.385223 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:41.385121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb" event={"ID":"6a4fc587-97d0-4fad-950f-3b1500773e2d","Type":"ContainerStarted","Data":"fb5286c5acf7e3072dd4666940f8ef6735e1ee1e163ad1b3d6ce6ee30a1be108"}
Sep 10 19:32:41.385577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:41.385220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb" event={"ID":"6a4fc587-97d0-4fad-950f-3b1500773e2d","Type":"ContainerStarted","Data":"b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5"}
Sep 10 19:32:41.404255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:41.404199    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb" podStartSLOduration=1.40418347 podStartE2EDuration="1.40418347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:32:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:32:41.402771001 +0000 UTC m=+1020.223827133" watchObservedRunningTime="2026-09-10 19:32:41.40418347 +0000 UTC m=+1020.225239603"
Sep 10 19:32:42.392656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:42.392624    2567 generic.go:356] "Generic (PLEG): container finished" podID="6a4fc587-97d0-4fad-950f-3b1500773e2d" containerID="fb5286c5acf7e3072dd4666940f8ef6735e1ee1e163ad1b3d6ce6ee30a1be108" exitCode=0
Sep 10 19:32:42.393101 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:42.392710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb" event={"ID":"6a4fc587-97d0-4fad-950f-3b1500773e2d","Type":"ContainerDied","Data":"fb5286c5acf7e3072dd4666940f8ef6735e1ee1e163ad1b3d6ce6ee30a1be108"}
Sep 10 19:32:43.537747 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:43.537723    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:43.575635 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:43.575604    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca\") pod \"6a4fc587-97d0-4fad-950f-3b1500773e2d\" (UID: \"6a4fc587-97d0-4fad-950f-3b1500773e2d\") "
Sep 10 19:32:43.575928 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:43.575907    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a4fc587-97d0-4fad-950f-3b1500773e2d" (UID: "6a4fc587-97d0-4fad-950f-3b1500773e2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:32:43.576052 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:43.576037    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4fc587-97d0-4fad-950f-3b1500773e2d-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:44.407055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:44.407023    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"
Sep 10 19:32:44.407237 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:44.407072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb" event={"ID":"6a4fc587-97d0-4fad-950f-3b1500773e2d","Type":"ContainerDied","Data":"b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5"}
Sep 10 19:32:44.407237 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:44.407123    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9bb7f980ee7b4732ef25b9b0ec380a49a4cdb1404fdcd6c8cd1fb809df4b7d5"
Sep 10 19:32:47.940968 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:47.940913    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:47.940968 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:47.940977    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:47.953890 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:47.953856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:48.441556 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:48.441525    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:32:50.997806 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:50.997767    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"]
Sep 10 19:32:51.001907 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:51.001873    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9a4e54-94hqb"]
Sep 10 19:32:51.707471 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:51.707439    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a4fc587-97d0-4fad-950f-3b1500773e2d" path="/var/lib/kubelet/pods/6a4fc587-97d0-4fad-950f-3b1500773e2d/volumes"
Sep 10 19:32:54.968284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:54.968246    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" podUID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" containerName="main" containerID="cri-o://c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574" gracePeriod=45
Sep 10 19:32:55.233558 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.233492    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:55.402945 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.402911    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403038    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403062    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403093    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403117    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r456z\" (UniqueName: \"kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403342 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403146    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm\") pod \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\" (UID: \"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8\") "
Sep 10 19:32:55.403402 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403350    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:55.403402 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403360    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache" (OuterVolumeSpecName: "model-cache") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:55.403402 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403380    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home" (OuterVolumeSpecName: "home") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:55.403555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403505    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.403555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403524    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.403555 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.403540    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.405248 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.405210    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm" (OuterVolumeSpecName: "dshm") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:32:55.405372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.405279    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:32:55.405372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.405333    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z" (OuterVolumeSpecName: "kube-api-access-r456z") pod "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" (UID: "f26b8f52-3e5a-4e73-8e9e-92e1aa516de8"). InnerVolumeSpecName "kube-api-access-r456z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:32:55.474920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.474886    2567 generic.go:356] "Generic (PLEG): container finished" podID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" containerID="c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574" exitCode=0
Sep 10 19:32:55.474920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.474928    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"
Sep 10 19:32:55.475151 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.474966    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" event={"ID":"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8","Type":"ContainerDied","Data":"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"}
Sep 10 19:32:55.475151 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.475018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg" event={"ID":"f26b8f52-3e5a-4e73-8e9e-92e1aa516de8","Type":"ContainerDied","Data":"60841328160f7aeb0c507399b4bd94c43ef5fe5883057f74b8ce782aeeef5417"}
Sep 10 19:32:55.475151 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.475039    2567 scope.go:117] "RemoveContainer" containerID="c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"
Sep 10 19:32:55.485348 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.485329    2567 scope.go:117] "RemoveContainer" containerID="c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"
Sep 10 19:32:55.485670 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:32:55.485655    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574\": container with ID starting with c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574 not found: ID does not exist" containerID="c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"
Sep 10 19:32:55.485720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.485682    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574"} err="failed to get container status \"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574\": rpc error: code = NotFound desc = could not find container \"c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574\": container with ID starting with c084a7b38499b718b19c41a6856df558abfa315f582f04c1a867388c9433c574 not found: ID does not exist"
Sep 10 19:32:55.497524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.497493    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:55.501828 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.501803    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-9a4e54/explicit-model-9a4e54-kserve-5d878cc897-pgsjg"]
Sep 10 19:32:55.504833 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.504801    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.504833 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.504836    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r456z\" (UniqueName: \"kubernetes.io/projected/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-kube-api-access-r456z\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.505004 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.504846    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:32:55.702484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:32:55.702446    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" path="/var/lib/kubelet/pods/f26b8f52-3e5a-4e73-8e9e-92e1aa516de8/volumes"
Sep 10 19:33:02.256446 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:02.256407    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:33:08.029920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.029883    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:08.030384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030352    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" containerName="main"
Sep 10 19:33:08.030384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" containerName="main"
Sep 10 19:33:08.030384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030378    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4fc587-97d0-4fad-950f-3b1500773e2d" containerName="revoke-keys"
Sep 10 19:33:08.030384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030384    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4fc587-97d0-4fad-950f-3b1500773e2d" containerName="revoke-keys"
Sep 10 19:33:08.030515 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030469    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f26b8f52-3e5a-4e73-8e9e-92e1aa516de8" containerName="main"
Sep 10 19:33:08.030515 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.030480    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4fc587-97d0-4fad-950f-3b1500773e2d" containerName="revoke-keys"
Sep 10 19:33:08.035142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.035121    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.039054 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.039028    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4c1277\"/\"openshift-service-ca.crt\""
Sep 10 19:33:08.039269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.039077    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4c1277\"/\"default-dockercfg-b4ncc\""
Sep 10 19:33:08.039269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.039142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4c1277\"/\"noten-model-4c1277-kserve-self-signed-certs\""
Sep 10 19:33:08.039269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.039249    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4c1277\"/\"kube-root-ca.crt\""
Sep 10 19:33:08.043353 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.043324    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:08.122253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.122449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.122449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.122449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.122449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122379    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.122662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.122461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhdbr\" (UniqueName: \"kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.223671 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhdbr\" (UniqueName: \"kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.223886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223740    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.223886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.223886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223804    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.224110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.224110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.223986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.224368 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.224337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.224474 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.224366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.224545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.224497    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.226395 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.226371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.226568 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.226546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.234085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.234060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhdbr\" (UniqueName: \"kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr\") pod \"noten-model-4c1277-kserve-ff457799f-qcb2d\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") " pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.346386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.346291    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:08.622846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:08.622815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:08.624696 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:33:08.624667    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2468b0b6_aa48_4c89_9483_cca05fb500d9.slice/crio-d8baa54377948c28741d339d08368d6caf315ae1f2738f3f01d74591967bbea5 WatchSource:0}: Error finding container d8baa54377948c28741d339d08368d6caf315ae1f2738f3f01d74591967bbea5: Status 404 returned error can't find the container with id d8baa54377948c28741d339d08368d6caf315ae1f2738f3f01d74591967bbea5
Sep 10 19:33:09.572604 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:09.572568    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" event={"ID":"2468b0b6-aa48-4c89-9483-cca05fb500d9","Type":"ContainerStarted","Data":"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"}
Sep 10 19:33:09.573039 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:09.572621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" event={"ID":"2468b0b6-aa48-4c89-9483-cca05fb500d9","Type":"ContainerStarted","Data":"d8baa54377948c28741d339d08368d6caf315ae1f2738f3f01d74591967bbea5"}
Sep 10 19:33:09.593888 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:09.593826    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" podStartSLOduration=1.593810331 podStartE2EDuration="1.593810331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:09.591195468 +0000 UTC m=+1048.412251616" watchObservedRunningTime="2026-09-10 19:33:09.593810331 +0000 UTC m=+1048.414866464"
Sep 10 19:33:16.726876 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:16.726841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:33:18.346751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:18.346710    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:18.347218 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:18.346764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:18.359171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:18.359144    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:18.639581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:18.639490    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:21.079571 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:21.077440    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:29.068240 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:29.068202    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:33:36.104418 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.104317    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" podUID="2468b0b6-aa48-4c89-9483-cca05fb500d9" containerName="main" containerID="cri-o://2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099" gracePeriod=45
Sep 10 19:33:36.360729 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.360669    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:36.479215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhdbr\" (UniqueName: \"kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479228    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479248    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479270    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479290    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479317    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home\") pod \"2468b0b6-aa48-4c89-9483-cca05fb500d9\" (UID: \"2468b0b6-aa48-4c89-9483-cca05fb500d9\") "
Sep 10 19:33:36.479695 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479523    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:36.479695 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479640    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:36.479796 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479702    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home" (OuterVolumeSpecName: "home") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:36.479796 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.479745    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.481713 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.481683    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:33:36.481713 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.481689    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr" (OuterVolumeSpecName: "kube-api-access-xhdbr") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "kube-api-access-xhdbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:33:36.481713 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.481699    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm" (OuterVolumeSpecName: "dshm") pod "2468b0b6-aa48-4c89-9483-cca05fb500d9" (UID: "2468b0b6-aa48-4c89-9483-cca05fb500d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:33:36.581298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.581259    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhdbr\" (UniqueName: \"kubernetes.io/projected/2468b0b6-aa48-4c89-9483-cca05fb500d9-kube-api-access-xhdbr\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.581298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.581289    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.581298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.581299    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2468b0b6-aa48-4c89-9483-cca05fb500d9-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.581298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.581308    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.581578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.581316    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2468b0b6-aa48-4c89-9483-cca05fb500d9-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:36.741675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.741643    2567 generic.go:356] "Generic (PLEG): container finished" podID="2468b0b6-aa48-4c89-9483-cca05fb500d9" containerID="2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099" exitCode=0
Sep 10 19:33:36.741850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.741683    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"
Sep 10 19:33:36.741850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.741707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" event={"ID":"2468b0b6-aa48-4c89-9483-cca05fb500d9","Type":"ContainerDied","Data":"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"}
Sep 10 19:33:36.741850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.741743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d" event={"ID":"2468b0b6-aa48-4c89-9483-cca05fb500d9","Type":"ContainerDied","Data":"d8baa54377948c28741d339d08368d6caf315ae1f2738f3f01d74591967bbea5"}
Sep 10 19:33:36.741850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.741762    2567 scope.go:117] "RemoveContainer" containerID="2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"
Sep 10 19:33:36.751943 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.751909    2567 scope.go:117] "RemoveContainer" containerID="2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"
Sep 10 19:33:36.752217 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:33:36.752199    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099\": container with ID starting with 2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099 not found: ID does not exist" containerID="2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"
Sep 10 19:33:36.752282 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.752228    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099"} err="failed to get container status \"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099\": rpc error: code = NotFound desc = could not find container \"2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099\": container with ID starting with 2591abce3c79313bea6fc25314975d8d08ef852de5e3aa20eeae038f92260099 not found: ID does not exist"
Sep 10 19:33:36.765787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.765759    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:36.774515 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:36.774493    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-4c1277/noten-model-4c1277-kserve-ff457799f-qcb2d"]
Sep 10 19:33:37.706490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:37.706444    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2468b0b6-aa48-4c89-9483-cca05fb500d9" path="/var/lib/kubelet/pods/2468b0b6-aa48-4c89-9483-cca05fb500d9/volumes"
Sep 10 19:33:51.103906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:51.103872    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:33:52.533554 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.533526    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:33:52.671734 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.671698    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"]
Sep 10 19:33:52.672142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.672129    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2468b0b6-aa48-4c89-9483-cca05fb500d9" containerName="main"
Sep 10 19:33:52.672201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.672143    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2468b0b6-aa48-4c89-9483-cca05fb500d9" containerName="main"
Sep 10 19:33:52.672238 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.672218    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2468b0b6-aa48-4c89-9483-cca05fb500d9" containerName="main"
Sep 10 19:33:52.676853 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.676837    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:52.681384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.681359    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:33:52.681535 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.681405    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:33:52.687189 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.687165    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"]
Sep 10 19:33:52.850785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.850695    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-173596-pkld7\" (UID: \"28f8f1c0-fe4c-4d07-8631-5c915feed165\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:52.951512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.951466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-173596-pkld7\" (UID: \"28f8f1c0-fe4c-4d07-8631-5c915feed165\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:52.952168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.952142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-173596-pkld7\" (UID: \"28f8f1c0-fe4c-4d07-8631-5c915feed165\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:52.989451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:52.989422    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:53.268845 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:53.268820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"]
Sep 10 19:33:53.270264 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:33:53.270226    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f8f1c0_fe4c_4d07_8631_5c915feed165.slice/crio-9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978 WatchSource:0}: Error finding container 9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978: Status 404 returned error can't find the container with id 9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978
Sep 10 19:33:53.844691 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:53.844604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7" event={"ID":"28f8f1c0-fe4c-4d07-8631-5c915feed165","Type":"ContainerStarted","Data":"d0bca6a352523088ef35a97ee63034c2beac00e1a0a366eb38ce84f4e145cbd1"}
Sep 10 19:33:53.844691 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:53.844655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7" event={"ID":"28f8f1c0-fe4c-4d07-8631-5c915feed165","Type":"ContainerStarted","Data":"9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978"}
Sep 10 19:33:53.865292 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:53.865238    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7" podStartSLOduration=1.8652188619999999 podStartE2EDuration="1.865218862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:33:53.863878363 +0000 UTC m=+1092.684934488" watchObservedRunningTime="2026-09-10 19:33:53.865218862 +0000 UTC m=+1092.686274989"
Sep 10 19:33:54.852580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:54.852542    2567 generic.go:356] "Generic (PLEG): container finished" podID="28f8f1c0-fe4c-4d07-8631-5c915feed165" containerID="d0bca6a352523088ef35a97ee63034c2beac00e1a0a366eb38ce84f4e145cbd1" exitCode=0
Sep 10 19:33:54.853008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:54.852628    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7" event={"ID":"28f8f1c0-fe4c-4d07-8631-5c915feed165","Type":"ContainerDied","Data":"d0bca6a352523088ef35a97ee63034c2beac00e1a0a366eb38ce84f4e145cbd1"}
Sep 10 19:33:55.849754 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:55.849716    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:33:55.999115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:55.999080    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:56.084377 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.084328    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca\") pod \"28f8f1c0-fe4c-4d07-8631-5c915feed165\" (UID: \"28f8f1c0-fe4c-4d07-8631-5c915feed165\") "
Sep 10 19:33:56.084692 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.084668    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "28f8f1c0-fe4c-4d07-8631-5c915feed165" (UID: "28f8f1c0-fe4c-4d07-8631-5c915feed165"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:33:56.084778 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.084758    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/28f8f1c0-fe4c-4d07-8631-5c915feed165-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:33:56.866967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.866913    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"
Sep 10 19:33:56.866967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.866962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7" event={"ID":"28f8f1c0-fe4c-4d07-8631-5c915feed165","Type":"ContainerDied","Data":"9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978"}
Sep 10 19:33:56.867175 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:33:56.867005    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9770a54b1381a5f68692c9dd965a0022e80623c2f94f82266b9d969f5d9e6978"
Sep 10 19:34:03.464587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.464551    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"]
Sep 10 19:34:03.472063 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.472033    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-173596-29817810-48jbw"]
Sep 10 19:34:03.523889 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.523850    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:34:03.524198 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.524150    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" podUID="c7147e22-23b9-4c8b-a48d-19550da8b571" containerName="payload-pre-processing" containerID="cri-o://33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd" gracePeriod=30
Sep 10 19:34:03.703303 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.703274    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74e1628e-4500-49b3-af1e-d1aaca494462" path="/var/lib/kubelet/pods/74e1628e-4500-49b3-af1e-d1aaca494462/volumes"
Sep 10 19:34:03.791018 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.790991    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:34:03.795354 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.795328    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"]
Sep 10 19:34:03.802520 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.802494    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-173596-pkld7"]
Sep 10 19:34:03.858514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.858482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume\") pod \"c7147e22-23b9-4c8b-a48d-19550da8b571\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") "
Sep 10 19:34:03.858681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.858606    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6n9r\" (UniqueName: \"kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r\") pod \"c7147e22-23b9-4c8b-a48d-19550da8b571\" (UID: \"c7147e22-23b9-4c8b-a48d-19550da8b571\") "
Sep 10 19:34:03.858957 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.858908    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7147e22-23b9-4c8b-a48d-19550da8b571" (UID: "c7147e22-23b9-4c8b-a48d-19550da8b571"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:03.859117 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.859101    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7147e22-23b9-4c8b-a48d-19550da8b571-plugins-config-volume\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:03.860770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.860750    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r" (OuterVolumeSpecName: "kube-api-access-q6n9r") pod "c7147e22-23b9-4c8b-a48d-19550da8b571" (UID: "c7147e22-23b9-4c8b-a48d-19550da8b571"). InnerVolumeSpecName "kube-api-access-q6n9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:03.908561 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.908523    2567 generic.go:356] "Generic (PLEG): container finished" podID="c7147e22-23b9-4c8b-a48d-19550da8b571" containerID="33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd" exitCode=0
Sep 10 19:34:03.908740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.908571    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"
Sep 10 19:34:03.908740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.908605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" event={"ID":"c7147e22-23b9-4c8b-a48d-19550da8b571","Type":"ContainerDied","Data":"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"}
Sep 10 19:34:03.908740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.908660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv" event={"ID":"c7147e22-23b9-4c8b-a48d-19550da8b571","Type":"ContainerDied","Data":"2fe7669c41411ac3bc8aebcd30acc758236500162045392612c415bfa76d2dd9"}
Sep 10 19:34:03.908740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.908684    2567 scope.go:117] "RemoveContainer" containerID="33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"
Sep 10 19:34:03.921779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.921753    2567 scope.go:117] "RemoveContainer" containerID="33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"
Sep 10 19:34:03.922339 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:03.922315    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd\": container with ID starting with 33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd not found: ID does not exist" containerID="33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"
Sep 10 19:34:03.922423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.922349    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd"} err="failed to get container status \"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd\": rpc error: code = NotFound desc = could not find container \"33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd\": container with ID starting with 33abb1dac61e1a8dac625c2f452919e6d11f26aae20cfbea4ee61614ca2398dd not found: ID does not exist"
Sep 10 19:34:03.937552 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.937520    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:34:03.944677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.944655    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-173596-558b9f698-pd2nv"]
Sep 10 19:34:03.959696 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:03.959674    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6n9r\" (UniqueName: \"kubernetes.io/projected/c7147e22-23b9-4c8b-a48d-19550da8b571-kube-api-access-q6n9r\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:05.702408 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:05.702377    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f8f1c0-fe4c-4d07-8631-5c915feed165" path="/var/lib/kubelet/pods/28f8f1c0-fe4c-4d07-8631-5c915feed165/volumes"
Sep 10 19:34:05.702787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:05.702728    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7147e22-23b9-4c8b-a48d-19550da8b571" path="/var/lib/kubelet/pods/c7147e22-23b9-4c8b-a48d-19550da8b571/volumes"
Sep 10 19:34:06.124052 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.123963    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" podUID="c994a894-12bc-4f53-9bd7-f6166adc01f3" containerName="main" containerID="cri-o://662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d" gracePeriod=45
Sep 10 19:34:06.383793 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.383725    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:34:06.484865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.484833    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.484902    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6qbx\" (UniqueName: \"kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.484926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.484972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485012    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home\") pod \"c994a894-12bc-4f53-9bd7-f6166adc01f3\" (UID: \"c994a894-12bc-4f53-9bd7-f6166adc01f3\") "
Sep 10 19:34:06.485365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485179    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:06.485365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:06.485485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485434    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.485485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485457    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.485485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.485431    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home" (OuterVolumeSpecName: "home") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:06.487307 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.487283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:34:06.487409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.487358    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm" (OuterVolumeSpecName: "dshm") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:06.487409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.487398    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx" (OuterVolumeSpecName: "kube-api-access-n6qbx") pod "c994a894-12bc-4f53-9bd7-f6166adc01f3" (UID: "c994a894-12bc-4f53-9bd7-f6166adc01f3"). InnerVolumeSpecName "kube-api-access-n6qbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:06.586494 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.586453    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.586494 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.586486    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c994a894-12bc-4f53-9bd7-f6166adc01f3-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.586494 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.586496    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6qbx\" (UniqueName: \"kubernetes.io/projected/c994a894-12bc-4f53-9bd7-f6166adc01f3-kube-api-access-n6qbx\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.586738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.586506    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c994a894-12bc-4f53-9bd7-f6166adc01f3-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:06.928361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.928322    2567 generic.go:356] "Generic (PLEG): container finished" podID="c994a894-12bc-4f53-9bd7-f6166adc01f3" containerID="662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d" exitCode=0
Sep 10 19:34:06.928883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.928382    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"
Sep 10 19:34:06.928883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.928403    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" event={"ID":"c994a894-12bc-4f53-9bd7-f6166adc01f3","Type":"ContainerDied","Data":"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"}
Sep 10 19:34:06.928883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.928450    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds" event={"ID":"c994a894-12bc-4f53-9bd7-f6166adc01f3","Type":"ContainerDied","Data":"c746befa31ed94e4af4a6bd13ac4d5b4b1fc3e065edc8f63dc5d763113ea96ce"}
Sep 10 19:34:06.928883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.928487    2567 scope.go:117] "RemoveContainer" containerID="662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"
Sep 10 19:34:06.938554 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.938537    2567 scope.go:117] "RemoveContainer" containerID="662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"
Sep 10 19:34:06.938809 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:06.938790    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d\": container with ID starting with 662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d not found: ID does not exist" containerID="662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"
Sep 10 19:34:06.938873 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.938822    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d"} err="failed to get container status \"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d\": rpc error: code = NotFound desc = could not find container \"662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d\": container with ID starting with 662a7f5755d4352594fd95f59f739d78db12966f52e3b07d9325af69fbbc374d not found: ID does not exist"
Sep 10 19:34:06.952269 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.952232    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:34:06.956699 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:06.956678    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-173596/test-model-173596-kserve-f5c4444c-wz8ds"]
Sep 10 19:34:07.553198 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.553152    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" podUID="c564ac30-6f96-4881-adaf-a168f2a92ad4" containerName="main" containerID="cri-o://21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873" gracePeriod=45
Sep 10 19:34:07.705079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.705044    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c994a894-12bc-4f53-9bd7-f6166adc01f3" path="/var/lib/kubelet/pods/c994a894-12bc-4f53-9bd7-f6166adc01f3/volumes"
Sep 10 19:34:07.813246 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.813181    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:34:07.834463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.834432    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:34:07.899422 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899384    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899455    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899524    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899596 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899573    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899766 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6cpj\" (UniqueName: \"kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj\") pod \"c564ac30-6f96-4881-adaf-a168f2a92ad4\" (UID: \"c564ac30-6f96-4881-adaf-a168f2a92ad4\") "
Sep 10 19:34:07.899766 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899681    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:07.899875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899781    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home" (OuterVolumeSpecName: "home") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:07.899875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.899836    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:07.900039 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.900023    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:07.900095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.900041    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:07.900095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.900055    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:07.901993 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.901959    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj" (OuterVolumeSpecName: "kube-api-access-x6cpj") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "kube-api-access-x6cpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:07.902236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.902216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm" (OuterVolumeSpecName: "dshm") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:07.902236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.902226    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c564ac30-6f96-4881-adaf-a168f2a92ad4" (UID: "c564ac30-6f96-4881-adaf-a168f2a92ad4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:34:07.936037 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.936004    2567 generic.go:356] "Generic (PLEG): container finished" podID="c564ac30-6f96-4881-adaf-a168f2a92ad4" containerID="21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873" exitCode=0
Sep 10 19:34:07.936425 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.936051    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"
Sep 10 19:34:07.936425 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.936094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" event={"ID":"c564ac30-6f96-4881-adaf-a168f2a92ad4","Type":"ContainerDied","Data":"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"}
Sep 10 19:34:07.936425 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.936145    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n" event={"ID":"c564ac30-6f96-4881-adaf-a168f2a92ad4","Type":"ContainerDied","Data":"f21a9e5674e6906a9e2478b890478639a4fe2740fc2b0e2851f731a9f74b59c9"}
Sep 10 19:34:07.936425 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.936164    2567 scope.go:117] "RemoveContainer" containerID="21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"
Sep 10 19:34:07.946070 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.946053    2567 scope.go:117] "RemoveContainer" containerID="21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"
Sep 10 19:34:07.946327 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:07.946310    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873\": container with ID starting with 21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873 not found: ID does not exist" containerID="21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"
Sep 10 19:34:07.946367 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.946338    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873"} err="failed to get container status \"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873\": rpc error: code = NotFound desc = could not find container \"21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873\": container with ID starting with 21d328a283a84b2fa963402c295331113c1d766a5776749131365d940a728873 not found: ID does not exist"
Sep 10 19:34:07.961888 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.961865    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:34:07.966368 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:07.966349    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e7e4c3/test-model-e7e4c3-kserve-84c785cb8c-2xb6n"]
Sep 10 19:34:08.001359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:08.001329    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c564ac30-6f96-4881-adaf-a168f2a92ad4-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:08.001359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:08.001355    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6cpj\" (UniqueName: \"kubernetes.io/projected/c564ac30-6f96-4881-adaf-a168f2a92ad4-kube-api-access-x6cpj\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:08.001359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:08.001365    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c564ac30-6f96-4881-adaf-a168f2a92ad4-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:09.702633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:09.702600    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c564ac30-6f96-4881-adaf-a168f2a92ad4" path="/var/lib/kubelet/pods/c564ac30-6f96-4881-adaf-a168f2a92ad4/volumes"
Sep 10 19:34:12.292231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292196    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"]
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292606    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c564ac30-6f96-4881-adaf-a168f2a92ad4" containerName="main"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292618    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c564ac30-6f96-4881-adaf-a168f2a92ad4" containerName="main"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292632    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f8f1c0-fe4c-4d07-8631-5c915feed165" containerName="revoke-keys"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292637    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f8f1c0-fe4c-4d07-8631-5c915feed165" containerName="revoke-keys"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292652    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c994a894-12bc-4f53-9bd7-f6166adc01f3" containerName="main"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292658    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c994a894-12bc-4f53-9bd7-f6166adc01f3" containerName="main"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292667    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7147e22-23b9-4c8b-a48d-19550da8b571" containerName="payload-pre-processing"
Sep 10 19:34:12.292708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292672    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7147e22-23b9-4c8b-a48d-19550da8b571" containerName="payload-pre-processing"
Sep 10 19:34:12.293145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292745    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f8f1c0-fe4c-4d07-8631-5c915feed165" containerName="revoke-keys"
Sep 10 19:34:12.293145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292757    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c564ac30-6f96-4881-adaf-a168f2a92ad4" containerName="main"
Sep 10 19:34:12.293145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292763    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c994a894-12bc-4f53-9bd7-f6166adc01f3" containerName="main"
Sep 10 19:34:12.293145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.292769    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7147e22-23b9-4c8b-a48d-19550da8b571" containerName="payload-pre-processing"
Sep 10 19:34:12.301718 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.301692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:12.305686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.305663    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:34:12.306201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.306179    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:34:12.310226 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.310205    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"]
Sep 10 19:34:12.447298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.447260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v\" (UID: \"7f1887ef-cc29-4293-a2b1-62e62c0c6243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:12.548314 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.548230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v\" (UID: \"7f1887ef-cc29-4293-a2b1-62e62c0c6243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:12.548866 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.548846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v\" (UID: \"7f1887ef-cc29-4293-a2b1-62e62c0c6243\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:12.614341 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.614306    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:12.896686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.896661    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"]
Sep 10 19:34:12.898978 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:34:12.898949    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1887ef_cc29_4293_a2b1_62e62c0c6243.slice/crio-4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c WatchSource:0}: Error finding container 4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c: Status 404 returned error can't find the container with id 4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c
Sep 10 19:34:12.969093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:12.969055    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v" event={"ID":"7f1887ef-cc29-4293-a2b1-62e62c0c6243","Type":"ContainerStarted","Data":"4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c"}
Sep 10 19:34:13.976451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:13.976419    2567 generic.go:356] "Generic (PLEG): container finished" podID="7f1887ef-cc29-4293-a2b1-62e62c0c6243" containerID="6022f59d41442dd46e2489dfa4d1aaa25635cf945bd3007d991848356d4b5c63" exitCode=0
Sep 10 19:34:13.976920 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:13.976507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v" event={"ID":"7f1887ef-cc29-4293-a2b1-62e62c0c6243","Type":"ContainerDied","Data":"6022f59d41442dd46e2489dfa4d1aaa25635cf945bd3007d991848356d4b5c63"}
Sep 10 19:34:15.117750 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.117724    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:15.173442 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.173415    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca\") pod \"7f1887ef-cc29-4293-a2b1-62e62c0c6243\" (UID: \"7f1887ef-cc29-4293-a2b1-62e62c0c6243\") "
Sep 10 19:34:15.173751 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.173727    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f1887ef-cc29-4293-a2b1-62e62c0c6243" (UID: "7f1887ef-cc29-4293-a2b1-62e62c0c6243"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:15.274877 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.274793    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1887ef-cc29-4293-a2b1-62e62c0c6243-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:15.991533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.991450    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"
Sep 10 19:34:15.991680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.991451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v" event={"ID":"7f1887ef-cc29-4293-a2b1-62e62c0c6243","Type":"ContainerDied","Data":"4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c"}
Sep 10 19:34:15.991680 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:15.991569    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4678b74eb66bfcc48135df12bb3de809874aed87a3e63392b46e73d1852ab69c"
Sep 10 19:34:20.417367 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.417335    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"]
Sep 10 19:34:20.418028 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.418009    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1887ef-cc29-4293-a2b1-62e62c0c6243" containerName="revoke-keys"
Sep 10 19:34:20.418098 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.418031    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1887ef-cc29-4293-a2b1-62e62c0c6243" containerName="revoke-keys"
Sep 10 19:34:20.418157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.418146    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f1887ef-cc29-4293-a2b1-62e62c0c6243" containerName="revoke-keys"
Sep 10 19:34:20.421721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.421694    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:20.424722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.424491    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:34:20.424870 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.424788    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:34:20.428178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.428142    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"]
Sep 10 19:34:20.525729 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.525696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh\" (UID: \"8ea2940f-aa1c-48de-8a02-165d76210a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:20.627091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.627050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh\" (UID: \"8ea2940f-aa1c-48de-8a02-165d76210a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:20.627705 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.627684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh\" (UID: \"8ea2940f-aa1c-48de-8a02-165d76210a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:20.734255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:20.734154    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:21.008810 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:21.008719    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"]
Sep 10 19:34:21.012630 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:34:21.012589    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea2940f_aa1c_48de_8a02_165d76210a6e.slice/crio-167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366 WatchSource:0}: Error finding container 167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366: Status 404 returned error can't find the container with id 167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366
Sep 10 19:34:21.025158 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:21.025130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh" event={"ID":"8ea2940f-aa1c-48de-8a02-165d76210a6e","Type":"ContainerStarted","Data":"167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366"}
Sep 10 19:34:22.031770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:22.031675    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ea2940f-aa1c-48de-8a02-165d76210a6e" containerID="a90cb984c78fba1f6312f0dc6ca9a45c29599a3be53628630c75353270d78064" exitCode=0
Sep 10 19:34:22.031770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:22.031720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh" event={"ID":"8ea2940f-aa1c-48de-8a02-165d76210a6e","Type":"ContainerDied","Data":"a90cb984c78fba1f6312f0dc6ca9a45c29599a3be53628630c75353270d78064"}
Sep 10 19:34:23.173006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.172982    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:23.252918 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.252882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca\") pod \"8ea2940f-aa1c-48de-8a02-165d76210a6e\" (UID: \"8ea2940f-aa1c-48de-8a02-165d76210a6e\") "
Sep 10 19:34:23.253355 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.253320    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ea2940f-aa1c-48de-8a02-165d76210a6e" (UID: "8ea2940f-aa1c-48de-8a02-165d76210a6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:23.317321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.316996    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:34:23.317321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.317240    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" podUID="73d41a67-f096-413b-8a4b-facefc6f2ea8" containerName="payload-pre-processing" containerID="cri-o://da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba" gracePeriod=30
Sep 10 19:34:23.354062 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.354025    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea2940f-aa1c-48de-8a02-165d76210a6e-maas-api-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:23.505517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.505484    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"]
Sep 10 19:34:23.508275 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.508246    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-e7e4c3-7jq9v"]
Sep 10 19:34:23.582309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.582246    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:34:23.702367 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.702335    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f1887ef-cc29-4293-a2b1-62e62c0c6243" path="/var/lib/kubelet/pods/7f1887ef-cc29-4293-a2b1-62e62c0c6243/volumes"
Sep 10 19:34:23.759517 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.759482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k6l7\" (UniqueName: \"kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7\") pod \"73d41a67-f096-413b-8a4b-facefc6f2ea8\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") "
Sep 10 19:34:23.759697 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.759563    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume\") pod \"73d41a67-f096-413b-8a4b-facefc6f2ea8\" (UID: \"73d41a67-f096-413b-8a4b-facefc6f2ea8\") "
Sep 10 19:34:23.759988 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.759963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73d41a67-f096-413b-8a4b-facefc6f2ea8" (UID: "73d41a67-f096-413b-8a4b-facefc6f2ea8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:23.761649 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.761626    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7" (OuterVolumeSpecName: "kube-api-access-7k6l7") pod "73d41a67-f096-413b-8a4b-facefc6f2ea8" (UID: "73d41a67-f096-413b-8a4b-facefc6f2ea8"). InnerVolumeSpecName "kube-api-access-7k6l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:23.860353 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.860263    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k6l7\" (UniqueName: \"kubernetes.io/projected/73d41a67-f096-413b-8a4b-facefc6f2ea8-kube-api-access-7k6l7\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:23.860353 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:23.860298    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73d41a67-f096-413b-8a4b-facefc6f2ea8-plugins-config-volume\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:24.046024 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.045984    2567 generic.go:356] "Generic (PLEG): container finished" podID="73d41a67-f096-413b-8a4b-facefc6f2ea8" containerID="da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba" exitCode=0
Sep 10 19:34:24.046220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.046042    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"
Sep 10 19:34:24.046220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.046074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" event={"ID":"73d41a67-f096-413b-8a4b-facefc6f2ea8","Type":"ContainerDied","Data":"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"}
Sep 10 19:34:24.046220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.046133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt" event={"ID":"73d41a67-f096-413b-8a4b-facefc6f2ea8","Type":"ContainerDied","Data":"500040782e2d3abc5583262be5bf489ffb89dfb58c312a8d9502664251c5f4e6"}
Sep 10 19:34:24.046220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.046156    2567 scope.go:117] "RemoveContainer" containerID="da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"
Sep 10 19:34:24.047679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.047602    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"
Sep 10 19:34:24.047679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.047634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh" event={"ID":"8ea2940f-aa1c-48de-8a02-165d76210a6e","Type":"ContainerDied","Data":"167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366"}
Sep 10 19:34:24.047821 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.047679    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="167bd48ceac797c7c79c5f4ccc99bd64e9479f834ae0235e60bd3a9a856a4366"
Sep 10 19:34:24.056579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.056558    2567 scope.go:117] "RemoveContainer" containerID="da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"
Sep 10 19:34:24.056867 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:24.056848    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba\": container with ID starting with da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba not found: ID does not exist" containerID="da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"
Sep 10 19:34:24.056951 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.056874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba"} err="failed to get container status \"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba\": rpc error: code = NotFound desc = could not find container \"da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba\": container with ID starting with da894b7464c03bb96e8f70a7cc9d755477bc5da57ae0f81defb1d14ce276f2ba not found: ID does not exist"
Sep 10 19:34:24.072915 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.072884    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:34:24.080435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:24.080409    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e7e4c3-75947c696b-kxwkt"]
Sep 10 19:34:25.703087 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:25.703045    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73d41a67-f096-413b-8a4b-facefc6f2ea8" path="/var/lib/kubelet/pods/73d41a67-f096-413b-8a4b-facefc6f2ea8/volumes"
Sep 10 19:34:26.250050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:26.250012    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:34:31.626825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.626795    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:34:31.627309 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.627030    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" containerID="cri-o://aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64" gracePeriod=30
Sep 10 19:34:31.703726 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.703689    2567 patch_prober.go:28] interesting pod/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 19:34:31.703913 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.703763    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 19:34:31.902249 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.902215    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"]
Sep 10 19:34:31.908031 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:31.908007    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cecfebe7-qwwkh"]
Sep 10 19:34:33.702883 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:33.702851    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea2940f-aa1c-48de-8a02-165d76210a6e" path="/var/lib/kubelet/pods/8ea2940f-aa1c-48de-8a02-165d76210a6e/volumes"
Sep 10 19:34:34.467405 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:34.467368    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:34:36.887879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.887853    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:34:36.981149 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981109    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981165    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981185    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gq28\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981217    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981250    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981303    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981327    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981359    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981724 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981417    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs\") pod \"0631a46a-3e07-4048-8b1f-639d0e0a958c\" (UID: \"0631a46a-3e07-4048-8b1f-639d0e0a958c\") "
Sep 10 19:34:36.981724 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981534    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:36.981724 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981584    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:34:36.981886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981814    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:36.981886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981827    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:36.981886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981867    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0631a46a-3e07-4048-8b1f-639d0e0a958c-istiod-ca-cert\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:36.982034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981887    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-credential-socket\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:36.982034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.981901    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:36.983580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.983543    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:36.983674 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.983623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 19:34:36.983925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.983910    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28" (OuterVolumeSpecName: "kube-api-access-2gq28") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "kube-api-access-2gq28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:34:36.983989 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:36.983924    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0631a46a-3e07-4048-8b1f-639d0e0a958c" (UID: "0631a46a-3e07-4048-8b1f-639d0e0a958c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:34:37.082926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082884    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-podinfo\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.082926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082915    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-token\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.082926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082924    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-data\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.082926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082966    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.083243 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082976    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gq28\" (UniqueName: \"kubernetes.io/projected/0631a46a-3e07-4048-8b1f-639d0e0a958c-kube-api-access-2gq28\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.083243 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082985    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-istio-envoy\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.083243 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.082994    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0631a46a-3e07-4048-8b1f-639d0e0a958c-workload-socket\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:34:37.129708 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.129670    2567 generic.go:356] "Generic (PLEG): container finished" podID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerID="aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64" exitCode=0
Sep 10 19:34:37.129849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.129720    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"
Sep 10 19:34:37.129849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.129760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerDied","Data":"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"}
Sep 10 19:34:37.129849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.129816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp" event={"ID":"0631a46a-3e07-4048-8b1f-639d0e0a958c","Type":"ContainerDied","Data":"dd9f915dc425dfe4a5d7b1af8e0e246c65a24f9c93cc17858caf89fb3cbacd10"}
Sep 10 19:34:37.129849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.129838    2567 scope.go:117] "RemoveContainer" containerID="aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"
Sep 10 19:34:37.140043 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.140021    2567 scope.go:117] "RemoveContainer" containerID="0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"
Sep 10 19:34:37.148844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.148825    2567 scope.go:117] "RemoveContainer" containerID="aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"
Sep 10 19:34:37.149173 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:37.149154    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64\": container with ID starting with aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64 not found: ID does not exist" containerID="aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"
Sep 10 19:34:37.149221 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.149184    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64"} err="failed to get container status \"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64\": rpc error: code = NotFound desc = could not find container \"aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64\": container with ID starting with aa7d1f1a83931e2cf2cda9116c9033274dfc23f198c8e7966e4136fa37ba7c64 not found: ID does not exist"
Sep 10 19:34:37.149221 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.149201    2567 scope.go:117] "RemoveContainer" containerID="0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"
Sep 10 19:34:37.149469 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:34:37.149450    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee\": container with ID starting with 0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee not found: ID does not exist" containerID="0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"
Sep 10 19:34:37.149521 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.149474    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee"} err="failed to get container status \"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee\": rpc error: code = NotFound desc = could not find container \"0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee\": container with ID starting with 0334a4b500e262833dcea35bc2bfb66191c9802acd12f656f0a492a5ea9a07ee not found: ID does not exist"
Sep 10 19:34:37.168033 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.168000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:34:37.184049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.184021    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e7e4c3-openshift-default-6cc8955ccd-mf4dp"]
Sep 10 19:34:37.702714 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:37.702677    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" path="/var/lib/kubelet/pods/0631a46a-3e07-4048-8b1f-639d0e0a958c/volumes"
Sep 10 19:34:40.887076 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:34:40.887038    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:35:05.550652 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.550550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:35:05.870802 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.870715    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:35:05.871145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871131    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea2940f-aa1c-48de-8a02-165d76210a6e" containerName="revoke-keys"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871147    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea2940f-aa1c-48de-8a02-165d76210a6e" containerName="revoke-keys"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871171    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871177    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871185    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871190    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871202    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73d41a67-f096-413b-8a4b-facefc6f2ea8" containerName="payload-pre-processing"
Sep 10 19:35:05.871207 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871208    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="73d41a67-f096-413b-8a4b-facefc6f2ea8" containerName="payload-pre-processing"
Sep 10 19:35:05.871423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871275    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea2940f-aa1c-48de-8a02-165d76210a6e" containerName="revoke-keys"
Sep 10 19:35:05.871423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871283    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="73d41a67-f096-413b-8a4b-facefc6f2ea8" containerName="payload-pre-processing"
Sep 10 19:35:05.871423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871288    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.871423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.871293    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0631a46a-3e07-4048-8b1f-639d0e0a958c" containerName="istio-proxy"
Sep 10 19:35:05.876137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.876118    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:05.878546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.878522    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gr54c\""
Sep 10 19:35:05.878677 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.878578    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:35:05.879591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.879540    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:35:05.879591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.879584    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-b83810-serving-cert\""
Sep 10 19:35:05.879771 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.879676    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:35:05.886636 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.886473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:35:05.961898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.961859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhv52\" (UniqueName: \"kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:05.962136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.961925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:05.962136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.962025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:05.962136 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:05.962097    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.063079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.063045    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhv52\" (UniqueName: \"kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.063256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.063094    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.063256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.063225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.063369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.063291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.063454 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:06.063435    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-serving-cert: secret "maas-api-e2e-shared-b-w3-b83810-serving-cert" not found
Sep 10 19:35:06.063534 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:06.063521    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls podName:78391b61-ba56-4006-8989-c3d620be146e nodeName:}" failed. No retries permitted until 2026-09-10 19:35:06.56349799 +0000 UTC m=+1165.384554106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls") pod "maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" (UID: "78391b61-ba56-4006-8989-c3d620be146e") : secret "maas-api-e2e-shared-b-w3-b83810-serving-cert" not found
Sep 10 19:35:06.064000 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.063978    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.066279 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.066256    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.078898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.078864    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhv52\" (UniqueName: \"kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.568740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.568702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.571104 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.571085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:06.789971 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:06.789910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:07.070792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:07.070766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:35:07.072846 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:07.072807    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78391b61_ba56_4006_8989_c3d620be146e.slice/crio-4d56f50cdac51bb3c91d5db911788c89af5ff0a6c6e260e0b52336e323652532 WatchSource:0}: Error finding container 4d56f50cdac51bb3c91d5db911788c89af5ff0a6c6e260e0b52336e323652532: Status 404 returned error can't find the container with id 4d56f50cdac51bb3c91d5db911788c89af5ff0a6c6e260e0b52336e323652532
Sep 10 19:35:07.074709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:07.074687    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:35:07.310709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:07.310673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" event={"ID":"78391b61-ba56-4006-8989-c3d620be146e","Type":"ContainerStarted","Data":"4d56f50cdac51bb3c91d5db911788c89af5ff0a6c6e260e0b52336e323652532"}
Sep 10 19:35:10.333572 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:10.333542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" event={"ID":"78391b61-ba56-4006-8989-c3d620be146e","Type":"ContainerStarted","Data":"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"}
Sep 10 19:35:10.334061 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:10.333607    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:10.353629 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:10.353573    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" podStartSLOduration=3.129740314 podStartE2EDuration="5.353558024s" totalImagesPullingTime="2.22381771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:10.351680393 +0000 UTC m=+1169.172736526" watchObservedRunningTime="2026-09-10 19:35:10.353558024 +0000 UTC m=+1169.174614157"
Sep 10 19:35:16.345321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:16.345292    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:35:20.239462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:20.239367    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:35:29.263625 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.263587    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:35:29.267369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.267345    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.269964 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.269919    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-7bfb5d-serving-cert\""
Sep 10 19:35:29.277329 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.277304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:35:29.384050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.384014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.384253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.384125    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.384253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.384175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.384253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.384212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhk89\" (UniqueName: \"kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.485355 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.485315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.485574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.485393    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.485574 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.485437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.485574 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:29.485534    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-serving-cert: secret "maas-api-e2e-shared-b-w2-7bfb5d-serving-cert" not found
Sep 10 19:35:29.485753 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.485576    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhk89\" (UniqueName: \"kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.485753 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:29.485608    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls podName:4fb68514-d23f-476b-a1ee-4060987e60fb nodeName:}" failed. No retries permitted until 2026-09-10 19:35:29.985586499 +0000 UTC m=+1188.806642612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls") pod "maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" (UID: "4fb68514-d23f-476b-a1ee-4060987e60fb") : secret "maas-api-e2e-shared-b-w2-7bfb5d-serving-cert" not found
Sep 10 19:35:29.486188 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.486163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.487973 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.487947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.496612 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.496587    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhk89\" (UniqueName: \"kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.990315 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.990275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:29.992771 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:29.992748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:30.180059 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:30.180022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:30.460813 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:30.460785    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:35:30.462877 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:30.462855    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb68514_d23f_476b_a1ee_4060987e60fb.slice/crio-cee3809999ced56eb7921e38d413d649d57f05fbc38d86dc17bd94570c41018f WatchSource:0}: Error finding container cee3809999ced56eb7921e38d413d649d57f05fbc38d86dc17bd94570c41018f: Status 404 returned error can't find the container with id cee3809999ced56eb7921e38d413d649d57f05fbc38d86dc17bd94570c41018f
Sep 10 19:35:31.466991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:31.466952    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" event={"ID":"4fb68514-d23f-476b-a1ee-4060987e60fb","Type":"ContainerStarted","Data":"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"}
Sep 10 19:35:31.467408 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:31.467017    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:31.467408 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:31.467038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" event={"ID":"4fb68514-d23f-476b-a1ee-4060987e60fb","Type":"ContainerStarted","Data":"cee3809999ced56eb7921e38d413d649d57f05fbc38d86dc17bd94570c41018f"}
Sep 10 19:35:31.486648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:31.486598    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" podStartSLOduration=2.025816409 podStartE2EDuration="2.486580791s" totalImagesPullingTime="460.764382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:31.485018007 +0000 UTC m=+1190.306074140" watchObservedRunningTime="2026-09-10 19:35:31.486580791 +0000 UTC m=+1190.307636925"
Sep 10 19:35:35.736211 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:35.736170    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:35:37.478216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:37.478186    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:35:40.737745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.737710    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:35:40.741663 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.741643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.744779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.744754    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"kube-root-ca.crt\""
Sep 10 19:35:40.744779 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.744772    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"openshift-service-ca.crt\""
Sep 10 19:35:40.744972 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.744757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"default-dockercfg-6gsm6\""
Sep 10 19:35:40.744972 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.744757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"auth-test-model-28d250-kserve-self-signed-certs\""
Sep 10 19:35:40.752458 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.752426    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:35:40.893985 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.893954    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.894155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.893994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.894155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.894016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8c55\" (UniqueName: \"kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.894155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.894063    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.894155 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.894141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.894310 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.894170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995519    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8c55\" (UniqueName: \"kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.995983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995929    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.996035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.995980    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.996035 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.996001    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.997994 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.997970    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:40.998344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:40.998325    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:41.005235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.005213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8c55\" (UniqueName: \"kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55\") pod \"auth-test-model-28d250-kserve-6894b84d7f-h4h5s\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") " pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:41.051177 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.051146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:41.329786 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.329760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:35:41.331500 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:41.331464    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fc455a3_f1f7_4951_8163_05a3e65e1460.slice/crio-5679b8fe6b37208d2037adfafda523a5694be6890fbf1cf814fe7175995a915e WatchSource:0}: Error finding container 5679b8fe6b37208d2037adfafda523a5694be6890fbf1cf814fe7175995a915e: Status 404 returned error can't find the container with id 5679b8fe6b37208d2037adfafda523a5694be6890fbf1cf814fe7175995a915e
Sep 10 19:35:41.527569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.527529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" event={"ID":"7fc455a3-f1f7-4951-8163-05a3e65e1460","Type":"ContainerStarted","Data":"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"}
Sep 10 19:35:41.527777 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.527580    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" event={"ID":"7fc455a3-f1f7-4951-8163-05a3e65e1460","Type":"ContainerStarted","Data":"5679b8fe6b37208d2037adfafda523a5694be6890fbf1cf814fe7175995a915e"}
Sep 10 19:35:41.548647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.548599    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" podStartSLOduration=1.548586432 podStartE2EDuration="1.548586432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:41.547878366 +0000 UTC m=+1200.368934500" watchObservedRunningTime="2026-09-10 19:35:41.548586432 +0000 UTC m=+1200.369642565"
Sep 10 19:35:41.735676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.735649    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:35:41.739690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.739662    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:35:41.742906 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.742885    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:35:41.747008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:41.746990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:35:49.437886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:49.437843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:35:51.051378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:51.051336    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:51.051799 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:51.051418    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:51.064051 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:51.064021    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:51.601573 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:51.601544    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:35:53.863974 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:53.863916    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8060-c8fcff"]
Sep 10 19:35:53.868689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:53.868663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:53.871312 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:53.871288    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bglfh\""
Sep 10 19:35:53.872189 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:53.872163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-c8fcff"]
Sep 10 19:35:54.019988 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.019919    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nmk2\" (UniqueName: \"kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2\") pod \"test-curl-8060-c8fcff\" (UID: \"94dfc44a-9799-4359-a338-e2eb286323cb\") " pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:54.121331 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.121252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nmk2\" (UniqueName: \"kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2\") pod \"test-curl-8060-c8fcff\" (UID: \"94dfc44a-9799-4359-a338-e2eb286323cb\") " pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:54.131109 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.131073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nmk2\" (UniqueName: \"kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2\") pod \"test-curl-8060-c8fcff\" (UID: \"94dfc44a-9799-4359-a338-e2eb286323cb\") " pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:54.180782 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.180745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:54.458911 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.458888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-c8fcff"]
Sep 10 19:35:54.460807 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:54.460780    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94dfc44a_9799_4359_a338_e2eb286323cb.slice/crio-372694ee999262a3e596824d0640a5a02c1cdcbd602afd13087fd2a11311383b WatchSource:0}: Error finding container 372694ee999262a3e596824d0640a5a02c1cdcbd602afd13087fd2a11311383b: Status 404 returned error can't find the container with id 372694ee999262a3e596824d0640a5a02c1cdcbd602afd13087fd2a11311383b
Sep 10 19:35:54.608366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.608330    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-c8fcff" event={"ID":"94dfc44a-9799-4359-a338-e2eb286323cb","Type":"ContainerStarted","Data":"c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e"}
Sep 10 19:35:54.608366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.608372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-c8fcff" event={"ID":"94dfc44a-9799-4359-a338-e2eb286323cb","Type":"ContainerStarted","Data":"372694ee999262a3e596824d0640a5a02c1cdcbd602afd13087fd2a11311383b"}
Sep 10 19:35:54.626145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:54.626092    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8060-c8fcff" podStartSLOduration=1.626076962 podStartE2EDuration="1.626076962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:54.623767475 +0000 UTC m=+1213.444823609" watchObservedRunningTime="2026-09-10 19:35:54.626076962 +0000 UTC m=+1213.447133094"
Sep 10 19:35:55.247075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.247032    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8060-c8fcff"]
Sep 10 19:35:55.251498 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.251471    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8060-c8fcff"]
Sep 10 19:35:55.373078 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.373038    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8060-16ae3b"]
Sep 10 19:35:55.373529 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.373510    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" containerName="test-curl-8060-c8fcff"
Sep 10 19:35:55.373618 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.373533    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" containerName="test-curl-8060-c8fcff"
Sep 10 19:35:55.373675 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.373635    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" containerName="test-curl-8060-c8fcff"
Sep 10 19:35:55.376837 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.376816    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:35:55.382886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.382853    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-16ae3b"]
Sep 10 19:35:55.533272 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.533170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-898dk\" (UniqueName: \"kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk\") pod \"test-curl-8060-16ae3b\" (UID: \"37abfe4b-516b-40d2-bdee-a0135bef192a\") " pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:35:55.614146 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:55.614115    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8060-c8fcff" container="test-curl-8060-c8fcff"
Sep 10 19:35:55.634181 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.634129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-898dk\" (UniqueName: \"kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk\") pod \"test-curl-8060-16ae3b\" (UID: \"37abfe4b-516b-40d2-bdee-a0135bef192a\") " pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:35:55.643463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.643430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-898dk\" (UniqueName: \"kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk\") pod \"test-curl-8060-16ae3b\" (UID: \"37abfe4b-516b-40d2-bdee-a0135bef192a\") " pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:35:55.697626 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.697599    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:35:55.970510 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:55.970484    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-16ae3b"]
Sep 10 19:35:55.972558 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:55.972528    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37abfe4b_516b_40d2_bdee_a0135bef192a.slice/crio-82dc779d27adb9b27d5c646801090d72d6d632ba027c6524cc3467a4a878b70c WatchSource:0}: Error finding container 82dc779d27adb9b27d5c646801090d72d6d632ba027c6524cc3467a4a878b70c: Status 404 returned error can't find the container with id 82dc779d27adb9b27d5c646801090d72d6d632ba027c6524cc3467a4a878b70c
Sep 10 19:35:56.621456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:56.621420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-16ae3b" event={"ID":"37abfe4b-516b-40d2-bdee-a0135bef192a","Type":"ContainerStarted","Data":"cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a"}
Sep 10 19:35:56.621456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:56.621462    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-16ae3b" event={"ID":"37abfe4b-516b-40d2-bdee-a0135bef192a","Type":"ContainerStarted","Data":"82dc779d27adb9b27d5c646801090d72d6d632ba027c6524cc3467a4a878b70c"}
Sep 10 19:35:56.621969 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:56.621590    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8060-c8fcff" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" containerName="test-curl-8060-c8fcff" containerID="cri-o://c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e" gracePeriod=2
Sep 10 19:35:56.641622 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:56.641575    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8060-16ae3b" podStartSLOduration=1.641560636 podStartE2EDuration="1.641560636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:56.638956103 +0000 UTC m=+1215.460012230" watchObservedRunningTime="2026-09-10 19:35:56.641560636 +0000 UTC m=+1215.462616769"
Sep 10 19:35:57.270523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.270487    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8060-16ae3b"]
Sep 10 19:35:57.276837 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.276810    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8060-16ae3b"]
Sep 10 19:35:57.401159 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.401127    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8060-c6094b"]
Sep 10 19:35:57.401541 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.401527    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" containerName="test-curl-8060-16ae3b"
Sep 10 19:35:57.401590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.401542    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" containerName="test-curl-8060-16ae3b"
Sep 10 19:35:57.401633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.401623    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" containerName="test-curl-8060-16ae3b"
Sep 10 19:35:57.404733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.404709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:35:57.409823 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.409797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-c6094b"]
Sep 10 19:35:57.552277 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.552245    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7hk\" (UniqueName: \"kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk\") pod \"test-curl-8060-c6094b\" (UID: \"f5d3978f-a24c-4e57-966a-113fa4e313dd\") " pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:35:57.627791 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:57.627761    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8060-16ae3b" container="test-curl-8060-16ae3b"
Sep 10 19:35:57.653194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.653125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7hk\" (UniqueName: \"kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk\") pod \"test-curl-8060-c6094b\" (UID: \"f5d3978f-a24c-4e57-966a-113fa4e313dd\") " pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:35:57.662414 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.662389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7hk\" (UniqueName: \"kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk\") pod \"test-curl-8060-c6094b\" (UID: \"f5d3978f-a24c-4e57-966a-113fa4e313dd\") " pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:35:57.715703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.715673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:35:57.988157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:57.988131    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-c6094b"]
Sep 10 19:35:57.989711 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:57.989680    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5d3978f_a24c_4e57_966a_113fa4e313dd.slice/crio-383a9dc3e2ddf84b6e41a42f05d56ce304619cf0d3fc14b1fd5c6e589270f401 WatchSource:0}: Error finding container 383a9dc3e2ddf84b6e41a42f05d56ce304619cf0d3fc14b1fd5c6e589270f401: Status 404 returned error can't find the container with id 383a9dc3e2ddf84b6e41a42f05d56ce304619cf0d3fc14b1fd5c6e589270f401
Sep 10 19:35:58.635465 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.635430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-c6094b" event={"ID":"f5d3978f-a24c-4e57-966a-113fa4e313dd","Type":"ContainerStarted","Data":"120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7"}
Sep 10 19:35:58.635842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.635483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-c6094b" event={"ID":"f5d3978f-a24c-4e57-966a-113fa4e313dd","Type":"ContainerStarted","Data":"383a9dc3e2ddf84b6e41a42f05d56ce304619cf0d3fc14b1fd5c6e589270f401"}
Sep 10 19:35:58.635842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.635609    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8060-16ae3b" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" containerName="test-curl-8060-16ae3b" containerID="cri-o://cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a" gracePeriod=2
Sep 10 19:35:58.651130 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.651083    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8060-c6094b" podStartSLOduration=1.6510641449999999 podStartE2EDuration="1.651064145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:35:58.650376156 +0000 UTC m=+1217.471432291" watchObservedRunningTime="2026-09-10 19:35:58.651064145 +0000 UTC m=+1217.472120280"
Sep 10 19:35:58.786752 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.786723    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:58.967752 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.967713    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nmk2\" (UniqueName: \"kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2\") pod \"94dfc44a-9799-4359-a338-e2eb286323cb\" (UID: \"94dfc44a-9799-4359-a338-e2eb286323cb\") "
Sep 10 19:35:58.969897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:58.969869    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2" (OuterVolumeSpecName: "kube-api-access-5nmk2") pod "94dfc44a-9799-4359-a338-e2eb286323cb" (UID: "94dfc44a-9799-4359-a338-e2eb286323cb"). InnerVolumeSpecName "kube-api-access-5nmk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:35:59.069505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.069473    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nmk2\" (UniqueName: \"kubernetes.io/projected/94dfc44a-9799-4359-a338-e2eb286323cb-kube-api-access-5nmk2\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:35:59.246123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.246026    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8060-c6094b"]
Sep 10 19:35:59.251295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.251272    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8060-c6094b"]
Sep 10 19:35:59.379409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.379371    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8060-eb481a"]
Sep 10 19:35:59.379796 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.379783    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" containerName="test-curl-8060-c6094b"
Sep 10 19:35:59.379842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.379797    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" containerName="test-curl-8060-c6094b"
Sep 10 19:35:59.379881 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.379864    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" containerName="test-curl-8060-c6094b"
Sep 10 19:35:59.383087 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.383071    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:35:59.389884 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.389857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-eb481a"]
Sep 10 19:35:59.474534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.474491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtqp9\" (UniqueName: \"kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9\") pod \"test-curl-8060-eb481a\" (UID: \"3867ae49-9d28-44a3-a033-fea34a58e60c\") " pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:35:59.575739 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.575639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtqp9\" (UniqueName: \"kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9\") pod \"test-curl-8060-eb481a\" (UID: \"3867ae49-9d28-44a3-a033-fea34a58e60c\") " pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:35:59.586388 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.586355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtqp9\" (UniqueName: \"kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9\") pod \"test-curl-8060-eb481a\" (UID: \"3867ae49-9d28-44a3-a033-fea34a58e60c\") " pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:35:59.642657 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.642620    2567 generic.go:356] "Generic (PLEG): container finished" podID="94dfc44a-9799-4359-a338-e2eb286323cb" containerID="c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e" exitCode=137
Sep 10 19:35:59.643118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.642664    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:59.643118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.642722    2567 scope.go:117] "RemoveContainer" containerID="c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e"
Sep 10 19:35:59.643118 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:59.643061    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8060-c6094b" container="test-curl-8060-c6094b"
Sep 10 19:35:59.652822 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.652804    2567 scope.go:117] "RemoveContainer" containerID="c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e"
Sep 10 19:35:59.653184 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:59.653166    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e\": container with ID starting with c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e not found: ID does not exist" containerID="c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e"
Sep 10 19:35:59.653184 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:35:59.653172    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8060-c8fcff\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" pod="opendatahub/test-curl-8060-c8fcff"
Sep 10 19:35:59.653281 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.653194    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e"} err="failed to get container status \"c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e\": rpc error: code = NotFound desc = could not find container \"c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e\": container with ID starting with c988d8b6899dcd9c5c31ea1bc40c9aafdfee7df9187b0919be5a7d77fe43805e not found: ID does not exist"
Sep 10 19:35:59.694211 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.694180    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:35:59.703858 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.703824    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94dfc44a-9799-4359-a338-e2eb286323cb" path="/var/lib/kubelet/pods/94dfc44a-9799-4359-a338-e2eb286323cb/volumes"
Sep 10 19:35:59.978169 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:35:59.978143    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8060-eb481a"]
Sep 10 19:35:59.980581 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:35:59.980555    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3867ae49_9d28_44a3_a033_fea34a58e60c.slice/crio-b0b96fb62327490275a26ca6c658dcec2d4d908056afad7766aed6d457e9f905 WatchSource:0}: Error finding container b0b96fb62327490275a26ca6c658dcec2d4d908056afad7766aed6d457e9f905: Status 404 returned error can't find the container with id b0b96fb62327490275a26ca6c658dcec2d4d908056afad7766aed6d457e9f905
Sep 10 19:36:00.654378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.654349    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-eb481a" event={"ID":"3867ae49-9d28-44a3-a033-fea34a58e60c","Type":"ContainerStarted","Data":"aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347"}
Sep 10 19:36:00.654770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.654387    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8060-eb481a" event={"ID":"3867ae49-9d28-44a3-a033-fea34a58e60c","Type":"ContainerStarted","Data":"b0b96fb62327490275a26ca6c658dcec2d4d908056afad7766aed6d457e9f905"}
Sep 10 19:36:00.654770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.654480    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8060-c6094b" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" containerName="test-curl-8060-c6094b" containerID="cri-o://120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7" gracePeriod=2
Sep 10 19:36:00.672531 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.672306    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8060-eb481a" podStartSLOduration=1.67228646 podStartE2EDuration="1.67228646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:00.669091334 +0000 UTC m=+1219.490147466" watchObservedRunningTime="2026-09-10 19:36:00.67228646 +0000 UTC m=+1219.493342592"
Sep 10 19:36:00.812514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.812487    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:36:00.988386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.988350    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-898dk\" (UniqueName: \"kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk\") pod \"37abfe4b-516b-40d2-bdee-a0135bef192a\" (UID: \"37abfe4b-516b-40d2-bdee-a0135bef192a\") "
Sep 10 19:36:00.990520 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:00.990490    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk" (OuterVolumeSpecName: "kube-api-access-898dk") pod "37abfe4b-516b-40d2-bdee-a0135bef192a" (UID: "37abfe4b-516b-40d2-bdee-a0135bef192a"). InnerVolumeSpecName "kube-api-access-898dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:01.089170 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.089141    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-898dk\" (UniqueName: \"kubernetes.io/projected/37abfe4b-516b-40d2-bdee-a0135bef192a-kube-api-access-898dk\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:36:01.282194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.282100    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8060-eb481a"]
Sep 10 19:36:01.290505 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.290467    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8060-eb481a"]
Sep 10 19:36:01.662162 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.662127    2567 generic.go:356] "Generic (PLEG): container finished" podID="37abfe4b-516b-40d2-bdee-a0135bef192a" containerID="cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a" exitCode=137
Sep 10 19:36:01.662644 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.662173    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:36:01.662644 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.662227    2567 scope.go:117] "RemoveContainer" containerID="cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a"
Sep 10 19:36:01.662644 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:01.662404    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8060-eb481a" container="test-curl-8060-eb481a"
Sep 10 19:36:01.672538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.672520    2567 scope.go:117] "RemoveContainer" containerID="cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a"
Sep 10 19:36:01.672869 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:01.672849    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a\": container with ID starting with cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a not found: ID does not exist" containerID="cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a"
Sep 10 19:36:01.672968 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.672882    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a"} err="failed to get container status \"cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a\": rpc error: code = NotFound desc = could not find container \"cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a\": container with ID starting with cbad48e38061f301e1d3aed8954815a98d494ed48db9ad0708fd4d29007b630a not found: ID does not exist"
Sep 10 19:36:01.673024 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:01.672970    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8060-16ae3b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:36:01.707150 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:01.707115    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8060-16ae3b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" pod="opendatahub/test-curl-8060-16ae3b"
Sep 10 19:36:01.709130 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:01.709108    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37abfe4b-516b-40d2-bdee-a0135bef192a" path="/var/lib/kubelet/pods/37abfe4b-516b-40d2-bdee-a0135bef192a/volumes"
Sep 10 19:36:02.669863 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:02.669829    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8060-eb481a" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" containerName="test-curl-8060-eb481a" containerID="cri-o://aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347" gracePeriod=2
Sep 10 19:36:02.846578 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:02.846547    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:36:03.010638 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.010533    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq7hk\" (UniqueName: \"kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk\") pod \"f5d3978f-a24c-4e57-966a-113fa4e313dd\" (UID: \"f5d3978f-a24c-4e57-966a-113fa4e313dd\") "
Sep 10 19:36:03.012679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.012644    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk" (OuterVolumeSpecName: "kube-api-access-zq7hk") pod "f5d3978f-a24c-4e57-966a-113fa4e313dd" (UID: "f5d3978f-a24c-4e57-966a-113fa4e313dd"). InnerVolumeSpecName "kube-api-access-zq7hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:03.111608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.111567    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq7hk\" (UniqueName: \"kubernetes.io/projected/f5d3978f-a24c-4e57-966a-113fa4e313dd-kube-api-access-zq7hk\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:36:03.676782 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.676747    2567 generic.go:356] "Generic (PLEG): container finished" podID="f5d3978f-a24c-4e57-966a-113fa4e313dd" containerID="120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7" exitCode=137
Sep 10 19:36:03.677296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.676797    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:36:03.677296 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.676812    2567 scope.go:117] "RemoveContainer" containerID="120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7"
Sep 10 19:36:03.687283 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.687262    2567 scope.go:117] "RemoveContainer" containerID="120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7"
Sep 10 19:36:03.687364 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:03.687260    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8060-c6094b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" pod="opendatahub/test-curl-8060-c6094b"
Sep 10 19:36:03.687580 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:03.687562    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7\": container with ID starting with 120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7 not found: ID does not exist" containerID="120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7"
Sep 10 19:36:03.687624 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.687589    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7"} err="failed to get container status \"120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7\": rpc error: code = NotFound desc = could not find container \"120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7\": container with ID starting with 120cf35f457c460dc0bbed046f39ad81d7866613e192b936bc614396204804c7 not found: ID does not exist"
Sep 10 19:36:03.702373 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:03.702342    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5d3978f-a24c-4e57-966a-113fa4e313dd" path="/var/lib/kubelet/pods/f5d3978f-a24c-4e57-966a-113fa4e313dd/volumes"
Sep 10 19:36:04.732577 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:04.732536    2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3867ae49_9d28_44a3_a033_fea34a58e60c.slice/crio-conmon-aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 19:36:04.852013 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:04.851986    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:36:04.924916 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:04.924831    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtqp9\" (UniqueName: \"kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9\") pod \"3867ae49-9d28-44a3-a033-fea34a58e60c\" (UID: \"3867ae49-9d28-44a3-a033-fea34a58e60c\") "
Sep 10 19:36:04.926948 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:04.926912    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9" (OuterVolumeSpecName: "kube-api-access-xtqp9") pod "3867ae49-9d28-44a3-a033-fea34a58e60c" (UID: "3867ae49-9d28-44a3-a033-fea34a58e60c"). InnerVolumeSpecName "kube-api-access-xtqp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:36:05.025997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.025960    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtqp9\" (UniqueName: \"kubernetes.io/projected/3867ae49-9d28-44a3-a033-fea34a58e60c-kube-api-access-xtqp9\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:36:05.691244 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.691209    2567 generic.go:356] "Generic (PLEG): container finished" podID="3867ae49-9d28-44a3-a033-fea34a58e60c" containerID="aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347" exitCode=137
Sep 10 19:36:05.691444 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.691250    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:36:05.691444 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.691276    2567 scope.go:117] "RemoveContainer" containerID="aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347"
Sep 10 19:36:05.700909 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.700889    2567 scope.go:117] "RemoveContainer" containerID="aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347"
Sep 10 19:36:05.701215 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:05.701195    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347\": container with ID starting with aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347 not found: ID does not exist" containerID="aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347"
Sep 10 19:36:05.701271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.701223    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347"} err="failed to get container status \"aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347\": rpc error: code = NotFound desc = could not find container \"aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347\": container with ID starting with aa4fb8fbc4143fbd36b42ea73979a17ab97eb3bd0d09cf71d8af1afbde6a6347 not found: ID does not exist"
Sep 10 19:36:05.701823 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:36:05.701799    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8060-eb481a\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" pod="opendatahub/test-curl-8060-eb481a"
Sep 10 19:36:05.702844 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:05.702825    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" path="/var/lib/kubelet/pods/3867ae49-9d28-44a3-a033-fea34a58e60c/volumes"
Sep 10 19:36:06.403300 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.403268    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:36:06.403709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.403696    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" containerName="test-curl-8060-eb481a"
Sep 10 19:36:06.403754 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.403710    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" containerName="test-curl-8060-eb481a"
Sep 10 19:36:06.403791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.403770    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3867ae49-9d28-44a3-a033-fea34a58e60c" containerName="test-curl-8060-eb481a"
Sep 10 19:36:06.408311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.408291    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.410396 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.410373    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-26a424d6-serving-cert\""
Sep 10 19:36:06.417417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.417393    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:36:06.438450 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.438419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.438621 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.438454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npc5s\" (UniqueName: \"kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.438621 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.438489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.438621 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.438566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.539863 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.539818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.540102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.539882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npc5s\" (UniqueName: \"kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.540102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.539982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.540102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.540055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.541060 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.541032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.544199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.544169    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.548128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.545103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.551072 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.551041    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npc5s\" (UniqueName: \"kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s\") pod \"maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:06.721216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:06.721123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:07.003276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:07.000183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:36:07.005334 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:36:07.005303    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06b4c36_620d_4982_91db_a5c5c83dc7ec.slice/crio-41b46305008b92e13ecd16b9a80049dd4a04a836b18ea239291a42f1435ba774 WatchSource:0}: Error finding container 41b46305008b92e13ecd16b9a80049dd4a04a836b18ea239291a42f1435ba774: Status 404 returned error can't find the container with id 41b46305008b92e13ecd16b9a80049dd4a04a836b18ea239291a42f1435ba774
Sep 10 19:36:07.706272 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:07.706234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" event={"ID":"f06b4c36-620d-4982-91db-a5c5c83dc7ec","Type":"ContainerStarted","Data":"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"}
Sep 10 19:36:07.706762 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:07.706290    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" event={"ID":"f06b4c36-620d-4982-91db-a5c5c83dc7ec","Type":"ContainerStarted","Data":"41b46305008b92e13ecd16b9a80049dd4a04a836b18ea239291a42f1435ba774"}
Sep 10 19:36:07.706762 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:07.706318    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:07.725470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:07.725410    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" podStartSLOduration=1.341773861 podStartE2EDuration="1.725392343s" totalImagesPullingTime="383.618482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:07.722790753 +0000 UTC m=+1226.543846888" watchObservedRunningTime="2026-09-10 19:36:07.725392343 +0000 UTC m=+1226.546448479"
Sep 10 19:36:13.719197 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:13.719163    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:36:14.525569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:14.525530    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:36:30.254590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.254557    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:36:30.259330 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.259311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.262551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.262388    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"openshift-service-ca.crt\""
Sep 10 19:36:30.262551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.262433    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"kube-root-ca.crt\""
Sep 10 19:36:30.262551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.262395    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"default-dockercfg-vnflj\""
Sep 10 19:36:30.262551 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.262395    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"auth-test-model-b83810-kserve-self-signed-certs\""
Sep 10 19:36:30.266290 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.266261    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:36:30.359540 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.359540 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.359770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.359770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359582    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.359770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.359770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.359720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9m5g\" (UniqueName: \"kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461267 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461225    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461267 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9m5g\" (UniqueName: \"kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461771 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461836 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.461898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.461851    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.463683 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.463665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.463967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.463949    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.469950 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.469912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9m5g\" (UniqueName: \"kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g\") pod \"auth-test-model-b83810-kserve-796c584795-cjrc2\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") " pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.571183 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.571090    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:30.851407 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:30.851383    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:36:30.853196 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:36:30.853172    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7a986df_8a40_4d87_a8ac_86f5f4233be5.slice/crio-daf8455868cc111c58f084ebec8b2cb436cb59999ccdabeb8555435601473e5a WatchSource:0}: Error finding container daf8455868cc111c58f084ebec8b2cb436cb59999ccdabeb8555435601473e5a: Status 404 returned error can't find the container with id daf8455868cc111c58f084ebec8b2cb436cb59999ccdabeb8555435601473e5a
Sep 10 19:36:31.860631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:31.860589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" event={"ID":"a7a986df-8a40-4d87-a8ac-86f5f4233be5","Type":"ContainerStarted","Data":"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"}
Sep 10 19:36:31.860631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:31.860636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" event={"ID":"a7a986df-8a40-4d87-a8ac-86f5f4233be5","Type":"ContainerStarted","Data":"daf8455868cc111c58f084ebec8b2cb436cb59999ccdabeb8555435601473e5a"}
Sep 10 19:36:31.882625 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:31.882571    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" podStartSLOduration=1.88255516 podStartE2EDuration="1.88255516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:36:31.881159611 +0000 UTC m=+1250.702215744" watchObservedRunningTime="2026-09-10 19:36:31.88255516 +0000 UTC m=+1250.703611316"
Sep 10 19:36:35.834200 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:35.834112    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:36:40.571312 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:40.571271    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:40.571312 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:40.571319    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:40.583703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:40.583671    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:40.942253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:40.942222    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:36:41.861659 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:41.861628    2567 scope.go:117] "RemoveContainer" containerID="741aeb418315cb73bf970d794003e816af792084b7616055f56522a5c2eb4ea1"
Sep 10 19:36:41.871006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:41.870986    2567 scope.go:117] "RemoveContainer" containerID="d4f70187a8b94c727ff069eeb23bb35d514883c0ebbe10bf7077d24b32e03323"
Sep 10 19:36:41.881807 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:36:41.881787    2567 scope.go:117] "RemoveContainer" containerID="476a700843983513abb26eccc44def399ad10121ac0729cb42834c495bf6b6ea"
Sep 10 19:37:08.828721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.828689    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:37:08.834044 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.834022    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.836605 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.836584    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-26a424d6\"/\"openshift-service-ca.crt\""
Sep 10 19:37:08.837442 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.837420    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-26a424d6\"/\"default-dockercfg-p7tg7\""
Sep 10 19:37:08.837518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.837434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-26a424d6\"/\"kube-root-ca.crt\""
Sep 10 19:37:08.837518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.837460    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-26a424d6\"/\"e2e-lifecycle-model-26a424d6-kserve-self-signed-certs\""
Sep 10 19:37:08.845082 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.845059    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:37:08.952070 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.952284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952086    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.952284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.952284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpn9b\" (UniqueName: \"kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.952456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:08.952456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:08.952340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.053749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.053712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.053971 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.053764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.053971 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.053802    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.053971 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.053828    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpn9b\" (UniqueName: \"kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.054174 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.053929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.054174 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.054019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.054317 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.054220    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.054317 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.054260    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.054436 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.054376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.056268 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.056246    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.056591 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.056571    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.063525 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.063504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpn9b\" (UniqueName: \"kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b\") pod \"e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") " pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.144466 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.144437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:09.419303 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:09.419279    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:37:09.421486 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:37:09.421457    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7b3c00a_67f0_4c6b_80b6_73b7786182a7.slice/crio-9e1e978d0f96bd9bf50022615810f6fde308c4fb4acb571ab6c412b41a27ca52 WatchSource:0}: Error finding container 9e1e978d0f96bd9bf50022615810f6fde308c4fb4acb571ab6c412b41a27ca52: Status 404 returned error can't find the container with id 9e1e978d0f96bd9bf50022615810f6fde308c4fb4acb571ab6c412b41a27ca52
Sep 10 19:37:10.110579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:10.110541    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" event={"ID":"f7b3c00a-67f0-4c6b-80b6-73b7786182a7","Type":"ContainerStarted","Data":"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"}
Sep 10 19:37:10.111013 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:10.110589    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" event={"ID":"f7b3c00a-67f0-4c6b-80b6-73b7786182a7","Type":"ContainerStarted","Data":"9e1e978d0f96bd9bf50022615810f6fde308c4fb4acb571ab6c412b41a27ca52"}
Sep 10 19:37:10.129875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:10.129823    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" podStartSLOduration=2.12980609 podStartE2EDuration="2.12980609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:37:10.128600689 +0000 UTC m=+1288.949656829" watchObservedRunningTime="2026-09-10 19:37:10.12980609 +0000 UTC m=+1288.950862222"
Sep 10 19:37:14.446201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:14.446160    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:37:19.145008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:19.144927    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:19.145008 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:19.145028    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:19.157846 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:19.157819    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:19.184864 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:19.184832    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:37:39.927472 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:39.927428    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:37:41.927137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:41.927103    2567 scope.go:117] "RemoveContainer" containerID="c75edff4073773dcc62f82f1357de49756c9030658f80691ce0ffc417952e430"
Sep 10 19:37:41.936196 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:41.936177    2567 scope.go:117] "RemoveContainer" containerID="1c0fd13c225dbcb08cea0b5bc041184c5d30baa0b147c2f38a7f94bdfaa5e6ce"
Sep 10 19:37:41.944902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:41.944881    2567 scope.go:117] "RemoveContainer" containerID="21e1e4f6ad70e30054db678ea802fde8f24bd1a740f75fb500568aca08131d95"
Sep 10 19:37:58.637930 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:37:58.637891    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:38:14.131564 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:14.131474    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:38:22.255450 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.255417    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:38:22.256011 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.255671    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" podUID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" containerName="maas-api" containerID="cri-o://9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23" gracePeriod=30
Sep 10 19:38:22.520148 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.520088    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:38:22.550713 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.550676    2567 generic.go:356] "Generic (PLEG): container finished" podID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" containerID="9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23" exitCode=0
Sep 10 19:38:22.550897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.550783    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" event={"ID":"f06b4c36-620d-4982-91db-a5c5c83dc7ec","Type":"ContainerDied","Data":"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"}
Sep 10 19:38:22.550897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.550791    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"
Sep 10 19:38:22.550897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.550837    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75" event={"ID":"f06b4c36-620d-4982-91db-a5c5c83dc7ec","Type":"ContainerDied","Data":"41b46305008b92e13ecd16b9a80049dd4a04a836b18ea239291a42f1435ba774"}
Sep 10 19:38:22.550897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.550870    2567 scope.go:117] "RemoveContainer" containerID="9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"
Sep 10 19:38:22.560797 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.560770    2567 scope.go:117] "RemoveContainer" containerID="9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"
Sep 10 19:38:22.561140 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:38:22.561119    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23\": container with ID starting with 9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23 not found: ID does not exist" containerID="9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"
Sep 10 19:38:22.561236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.561149    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23"} err="failed to get container status \"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23\": rpc error: code = NotFound desc = could not find container \"9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23\": container with ID starting with 9ccce073b769745ef543ebf32d28fb668fc9e2bbaf92268432fecf162fce5e23 not found: ID does not exist"
Sep 10 19:38:22.572569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.572542    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls\") pod \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") "
Sep 10 19:38:22.572706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.572586    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npc5s\" (UniqueName: \"kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s\") pod \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") "
Sep 10 19:38:22.572706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.572610    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca\") pod \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") "
Sep 10 19:38:22.572706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.572642    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs\") pod \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\" (UID: \"f06b4c36-620d-4982-91db-a5c5c83dc7ec\") "
Sep 10 19:38:22.573084 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.573057    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f06b4c36-620d-4982-91db-a5c5c83dc7ec" (UID: "f06b4c36-620d-4982-91db-a5c5c83dc7ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:38:22.573186 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.573172    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f06b4c36-620d-4982-91db-a5c5c83dc7ec-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.574884 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.574855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f06b4c36-620d-4982-91db-a5c5c83dc7ec" (UID: "f06b4c36-620d-4982-91db-a5c5c83dc7ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:22.575190 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.575169    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f06b4c36-620d-4982-91db-a5c5c83dc7ec" (UID: "f06b4c36-620d-4982-91db-a5c5c83dc7ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:22.575263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.575191    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s" (OuterVolumeSpecName: "kube-api-access-npc5s") pod "f06b4c36-620d-4982-91db-a5c5c83dc7ec" (UID: "f06b4c36-620d-4982-91db-a5c5c83dc7ec"). InnerVolumeSpecName "kube-api-access-npc5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:22.674038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.674011    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.674038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.674035    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npc5s\" (UniqueName: \"kubernetes.io/projected/f06b4c36-620d-4982-91db-a5c5c83dc7ec-kube-api-access-npc5s\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.674038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.674044    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f06b4c36-620d-4982-91db-a5c5c83dc7ec-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:22.877105 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.877070    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:38:22.879442 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:22.879421    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-26a424d6-6b658c58f6-j2s75"]
Sep 10 19:38:23.702597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:23.702563    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" path="/var/lib/kubelet/pods/f06b4c36-620d-4982-91db-a5c5c83dc7ec/volumes"
Sep 10 19:38:30.857291 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:30.857255    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:38:32.532950 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:32.532907    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:38:41.978168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:41.978136    2567 scope.go:117] "RemoveContainer" containerID="e8925c08cea6a122628ab6d7613b04a0838cef45a438f760be634f4e7840cd1c"
Sep 10 19:38:41.987080 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:41.987061    2567 scope.go:117] "RemoveContainer" containerID="fb5286c5acf7e3072dd4666940f8ef6735e1ee1e163ad1b3d6ce6ee30a1be108"
Sep 10 19:38:45.877372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:45.877310    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" podUID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" containerName="main" containerID="cri-o://17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556" gracePeriod=45
Sep 10 19:38:46.132544 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.132475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:38:46.313738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.313706    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpn9b\" (UniqueName: \"kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.313954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.313760    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.313954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.313801    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.313954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.313904    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.314146 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.313980    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.314146 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314039    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache\") pod \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\" (UID: \"f7b3c00a-67f0-4c6b-80b6-73b7786182a7\") "
Sep 10 19:38:46.314146 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314069    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home" (OuterVolumeSpecName: "home") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:46.314146 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314081    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:46.314339 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314290    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:46.314482 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314465    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.314543 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314487    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.314543 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.314502    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.316151 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.316116    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm" (OuterVolumeSpecName: "dshm") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:38:46.316261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.316175    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:38:46.316261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.316189    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b" (OuterVolumeSpecName: "kube-api-access-bpn9b") pod "f7b3c00a-67f0-4c6b-80b6-73b7786182a7" (UID: "f7b3c00a-67f0-4c6b-80b6-73b7786182a7"). InnerVolumeSpecName "kube-api-access-bpn9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:38:46.415217 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.415187    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.415217 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.415219    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.415426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.415233    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpn9b\" (UniqueName: \"kubernetes.io/projected/f7b3c00a-67f0-4c6b-80b6-73b7786182a7-kube-api-access-bpn9b\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:38:46.691749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.691659    2567 generic.go:356] "Generic (PLEG): container finished" podID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" containerID="17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556" exitCode=0
Sep 10 19:38:46.691749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.691706    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"
Sep 10 19:38:46.691945 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.691744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" event={"ID":"f7b3c00a-67f0-4c6b-80b6-73b7786182a7","Type":"ContainerDied","Data":"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"}
Sep 10 19:38:46.691945 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.691788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2" event={"ID":"f7b3c00a-67f0-4c6b-80b6-73b7786182a7","Type":"ContainerDied","Data":"9e1e978d0f96bd9bf50022615810f6fde308c4fb4acb571ab6c412b41a27ca52"}
Sep 10 19:38:46.691945 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.691805    2567 scope.go:117] "RemoveContainer" containerID="17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"
Sep 10 19:38:46.702847 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.702829    2567 scope.go:117] "RemoveContainer" containerID="17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"
Sep 10 19:38:46.703197 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:38:46.703174    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556\": container with ID starting with 17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556 not found: ID does not exist" containerID="17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"
Sep 10 19:38:46.703318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.703204    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556"} err="failed to get container status \"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556\": rpc error: code = NotFound desc = could not find container \"17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556\": container with ID starting with 17182295e8afd9e463433f251ca439edd5cbcfb22a6a2025f61743dae60cf556 not found: ID does not exist"
Sep 10 19:38:46.715360 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.715321    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:38:46.722392 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:46.722362    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-26a424d6/e2e-lifecycle-model-26a424d6-kserve-54d5dff4f5-bcrq2"]
Sep 10 19:38:47.343745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:47.343706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:38:47.701842 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:38:47.701805    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" path="/var/lib/kubelet/pods/f7b3c00a-67f0-4c6b-80b6-73b7786182a7/volumes"
Sep 10 19:39:04.027328 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:04.027288    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:39:22.235587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:22.235550    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:39:34.496811 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.496722    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8063-630997"]
Sep 10 19:39:34.497404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497332    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" containerName="main"
Sep 10 19:39:34.497404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497356    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" containerName="main"
Sep 10 19:39:34.497404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497401    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" containerName="maas-api"
Sep 10 19:39:34.497579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497410    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" containerName="maas-api"
Sep 10 19:39:34.497579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497543    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f06b4c36-620d-4982-91db-a5c5c83dc7ec" containerName="maas-api"
Sep 10 19:39:34.497579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.497562    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b3c00a-67f0-4c6b-80b6-73b7786182a7" containerName="main"
Sep 10 19:39:34.500734 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.500714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:34.505733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.505713    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bglfh\""
Sep 10 19:39:34.510478 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.510456    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-630997"]
Sep 10 19:39:34.582178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.582142    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmjnm\" (UniqueName: \"kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm\") pod \"test-curl-8063-630997\" (UID: \"e542f55e-8c73-4686-b724-1ddde4747a77\") " pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:34.683550 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.683503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmjnm\" (UniqueName: \"kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm\") pod \"test-curl-8063-630997\" (UID: \"e542f55e-8c73-4686-b724-1ddde4747a77\") " pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:34.694529 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.694488    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmjnm\" (UniqueName: \"kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm\") pod \"test-curl-8063-630997\" (UID: \"e542f55e-8c73-4686-b724-1ddde4747a77\") " pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:34.810852 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:34.810767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:35.086114 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:35.086088    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-630997"]
Sep 10 19:39:35.087536 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:39:35.087505    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode542f55e_8c73_4686_b724_1ddde4747a77.slice/crio-2deb373ee55ae64e5e26f50a7659b56d291f86dbac145f7bdc7d89d01718e55f WatchSource:0}: Error finding container 2deb373ee55ae64e5e26f50a7659b56d291f86dbac145f7bdc7d89d01718e55f: Status 404 returned error can't find the container with id 2deb373ee55ae64e5e26f50a7659b56d291f86dbac145f7bdc7d89d01718e55f
Sep 10 19:39:35.994687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:35.994636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-630997" event={"ID":"e542f55e-8c73-4686-b724-1ddde4747a77","Type":"ContainerStarted","Data":"f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68"}
Sep 10 19:39:35.994687 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:35.994694    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-630997" event={"ID":"e542f55e-8c73-4686-b724-1ddde4747a77","Type":"ContainerStarted","Data":"2deb373ee55ae64e5e26f50a7659b56d291f86dbac145f7bdc7d89d01718e55f"}
Sep 10 19:39:36.011681 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:36.011625    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8063-630997" podStartSLOduration=2.011608688 podStartE2EDuration="2.011608688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:36.009610787 +0000 UTC m=+1434.830666919" watchObservedRunningTime="2026-09-10 19:39:36.011608688 +0000 UTC m=+1434.832664820"
Sep 10 19:39:36.632526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:36.632491    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8063-630997"]
Sep 10 19:39:36.635959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:36.635921    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8063-630997"]
Sep 10 19:39:36.939031 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:36.938999    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:39:37.000399 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:37.000367    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8063-630997" container="test-curl-8063-630997"
Sep 10 19:39:38.006090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:38.006049    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8063-630997" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" containerName="test-curl-8063-630997" containerID="cri-o://f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68" gracePeriod=2
Sep 10 19:39:38.064215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:38.064184    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:39:39.302104 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:39.302070    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:39:40.196214 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:40.196186    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:40.236576 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:40.236544    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmjnm\" (UniqueName: \"kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm\") pod \"e542f55e-8c73-4686-b724-1ddde4747a77\" (UID: \"e542f55e-8c73-4686-b724-1ddde4747a77\") "
Sep 10 19:39:40.238650 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:40.238620    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm" (OuterVolumeSpecName: "kube-api-access-zmjnm") pod "e542f55e-8c73-4686-b724-1ddde4747a77" (UID: "e542f55e-8c73-4686-b724-1ddde4747a77"). InnerVolumeSpecName "kube-api-access-zmjnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:40.337554 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:40.337458    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmjnm\" (UniqueName: \"kubernetes.io/projected/e542f55e-8c73-4686-b724-1ddde4747a77-kube-api-access-zmjnm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:41.025144 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.025107    2567 generic.go:356] "Generic (PLEG): container finished" podID="e542f55e-8c73-4686-b724-1ddde4747a77" containerID="f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68" exitCode=137
Sep 10 19:39:41.025144 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.025141    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:41.025416 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.025206    2567 scope.go:117] "RemoveContainer" containerID="f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68"
Sep 10 19:39:41.035159 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:41.035031    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-630997\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:41.035231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.035166    2567 scope.go:117] "RemoveContainer" containerID="f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68"
Sep 10 19:39:41.035476 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:41.035458    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68\": container with ID starting with f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68 not found: ID does not exist" containerID="f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68"
Sep 10 19:39:41.035537 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.035490    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68"} err="failed to get container status \"f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68\": rpc error: code = NotFound desc = could not find container \"f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68\": container with ID starting with f593c8a8c6936fea0a5d0c9e304994f9fb88480f22ae4def20a09f06c76a8a68 not found: ID does not exist"
Sep 10 19:39:41.701170 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:41.701128    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-630997\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" pod="opendatahub/test-curl-8063-630997"
Sep 10 19:39:41.703096 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:41.703075    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" path="/var/lib/kubelet/pods/e542f55e-8c73-4686-b724-1ddde4747a77/volumes"
Sep 10 19:39:45.722305 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.722274    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:39:45.722712 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.722677    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" containerName="test-curl-8063-630997"
Sep 10 19:39:45.722712 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.722690    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" containerName="test-curl-8063-630997"
Sep 10 19:39:45.722794 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.722764    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e542f55e-8c73-4686-b724-1ddde4747a77" containerName="test-curl-8063-630997"
Sep 10 19:39:45.725912 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.725895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.728388 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.728367    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"sub-test-model-28d250-kserve-self-signed-certs\""
Sep 10 19:39:45.736980 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.736957    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:39:45.791682 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.791682 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.791947 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.791947 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzcf9\" (UniqueName: \"kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.791947 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.792131 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.791980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.892840 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.892788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.892840 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.892847    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.892866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.892895    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.892912    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzcf9\" (UniqueName: \"kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.893014    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.893343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893507 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.893402    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.893507 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.893416    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.895151 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.895133    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.895468 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.895450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:45.902542 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:45.902525    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzcf9\" (UniqueName: \"kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9\") pod \"sub-test-model-28d250-kserve-7d8d87567c-42c8w\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") " pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:46.035199 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:46.035098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:46.307608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:46.307582    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:39:46.309234 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:39:46.309205    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31b6f70c_a149_4d88_92e2_bbc4d9c755e8.slice/crio-ed7c0ce84599403882c9bf391820690906448586eff785e52591caf4e0ee92a5 WatchSource:0}: Error finding container ed7c0ce84599403882c9bf391820690906448586eff785e52591caf4e0ee92a5: Status 404 returned error can't find the container with id ed7c0ce84599403882c9bf391820690906448586eff785e52591caf4e0ee92a5
Sep 10 19:39:47.064066 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:47.064028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" event={"ID":"31b6f70c-a149-4d88-92e2-bbc4d9c755e8","Type":"ContainerStarted","Data":"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"}
Sep 10 19:39:47.064066 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:47.064072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" event={"ID":"31b6f70c-a149-4d88-92e2-bbc4d9c755e8","Type":"ContainerStarted","Data":"ed7c0ce84599403882c9bf391820690906448586eff785e52591caf4e0ee92a5"}
Sep 10 19:39:47.087057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:47.087002    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" podStartSLOduration=2.086985304 podStartE2EDuration="2.086985304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:39:47.084773392 +0000 UTC m=+1445.905829527" watchObservedRunningTime="2026-09-10 19:39:47.086985304 +0000 UTC m=+1445.908041457"
Sep 10 19:39:50.729864 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:50.729827    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:39:53.084346 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.084280    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" podUID="7fc455a3-f1f7-4951-8163-05a3e65e1460" containerName="main" containerID="cri-o://082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0" gracePeriod=45
Sep 10 19:39:53.349632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.349569    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:39:53.470586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470548    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.470767 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470603    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8c55\" (UniqueName: \"kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.470767 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470636    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.470767 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470666    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.470767 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470724    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.471029 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470865    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs\") pod \"7fc455a3-f1f7-4951-8163-05a3e65e1460\" (UID: \"7fc455a3-f1f7-4951-8163-05a3e65e1460\") "
Sep 10 19:39:53.471029 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470896    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache" (OuterVolumeSpecName: "model-cache") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:53.471029 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.470917    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home" (OuterVolumeSpecName: "home") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:53.471178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.471038    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:53.471288 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.471269    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:53.471346 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.471294    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:53.471346 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.471308    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:53.473019 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.472995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm" (OuterVolumeSpecName: "dshm") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:53.473127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.473020    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:53.473127 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.473062    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55" (OuterVolumeSpecName: "kube-api-access-g8c55") pod "7fc455a3-f1f7-4951-8163-05a3e65e1460" (UID: "7fc455a3-f1f7-4951-8163-05a3e65e1460"). InnerVolumeSpecName "kube-api-access-g8c55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:53.572179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.572142    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fc455a3-f1f7-4951-8163-05a3e65e1460-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:53.572179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.572171    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fc455a3-f1f7-4951-8163-05a3e65e1460-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:53.572179 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:53.572183    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8c55\" (UniqueName: \"kubernetes.io/projected/7fc455a3-f1f7-4951-8163-05a3e65e1460-kube-api-access-g8c55\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.106676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.106582    2567 generic.go:356] "Generic (PLEG): container finished" podID="7fc455a3-f1f7-4951-8163-05a3e65e1460" containerID="082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0" exitCode=0
Sep 10 19:39:54.106676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.106626    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"
Sep 10 19:39:54.106676 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.106652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" event={"ID":"7fc455a3-f1f7-4951-8163-05a3e65e1460","Type":"ContainerDied","Data":"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"}
Sep 10 19:39:54.107227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.106699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s" event={"ID":"7fc455a3-f1f7-4951-8163-05a3e65e1460","Type":"ContainerDied","Data":"5679b8fe6b37208d2037adfafda523a5694be6890fbf1cf814fe7175995a915e"}
Sep 10 19:39:54.107227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.106717    2567 scope.go:117] "RemoveContainer" containerID="082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"
Sep 10 19:39:54.115730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.115709    2567 scope.go:117] "RemoveContainer" containerID="082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"
Sep 10 19:39:54.116059 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:54.116034    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0\": container with ID starting with 082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0 not found: ID does not exist" containerID="082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"
Sep 10 19:39:54.116168 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.116071    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0"} err="failed to get container status \"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0\": rpc error: code = NotFound desc = could not find container \"082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0\": container with ID starting with 082f03f9258ed515fbd46c30ca533acc8d4a979fe824989441a25161ceb152a0 not found: ID does not exist"
Sep 10 19:39:54.127749 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.127719    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:39:54.130002 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.129982    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/auth-test-model-28d250-kserve-6894b84d7f-h4h5s"]
Sep 10 19:39:54.322447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.322408    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" podUID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" containerName="main" containerID="cri-o://805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463" gracePeriod=45
Sep 10 19:39:54.580662 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.580640    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:39:54.684032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.683984    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9m5g\" (UniqueName: \"kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684078    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684109    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684127    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684184    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684272    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home\") pod \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\" (UID: \"a7a986df-8a40-4d87-a8ac-86f5f4233be5\") "
Sep 10 19:39:54.684462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684421    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache" (OuterVolumeSpecName: "model-cache") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:54.684567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684484    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home" (OuterVolumeSpecName: "home") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:54.684567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:54.684686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684662    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.684686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684678    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.684787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.684691    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.686420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.686389    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g" (OuterVolumeSpecName: "kube-api-access-f9m5g") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "kube-api-access-f9m5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:39:54.686420 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.686410    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm" (OuterVolumeSpecName: "dshm") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:39:54.686584 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.686440    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a7a986df-8a40-4d87-a8ac-86f5f4233be5" (UID: "a7a986df-8a40-4d87-a8ac-86f5f4233be5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:39:54.786485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.786440    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7a986df-8a40-4d87-a8ac-86f5f4233be5-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.786485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.786470    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7a986df-8a40-4d87-a8ac-86f5f4233be5-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:54.786485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:54.786483    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9m5g\" (UniqueName: \"kubernetes.io/projected/a7a986df-8a40-4d87-a8ac-86f5f4233be5-kube-api-access-f9m5g\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:39:55.114384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.114296    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" containerID="805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463" exitCode=0
Sep 10 19:39:55.114384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.114342    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"
Sep 10 19:39:55.114903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.114381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" event={"ID":"a7a986df-8a40-4d87-a8ac-86f5f4233be5","Type":"ContainerDied","Data":"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"}
Sep 10 19:39:55.114903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.114420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2" event={"ID":"a7a986df-8a40-4d87-a8ac-86f5f4233be5","Type":"ContainerDied","Data":"daf8455868cc111c58f084ebec8b2cb436cb59999ccdabeb8555435601473e5a"}
Sep 10 19:39:55.114903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.114442    2567 scope.go:117] "RemoveContainer" containerID="805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"
Sep 10 19:39:55.124064 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.124041    2567 scope.go:117] "RemoveContainer" containerID="805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"
Sep 10 19:39:55.124346 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:39:55.124328    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463\": container with ID starting with 805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463 not found: ID does not exist" containerID="805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"
Sep 10 19:39:55.124410 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.124358    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463"} err="failed to get container status \"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463\": rpc error: code = NotFound desc = could not find container \"805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463\": container with ID starting with 805d1e3fb734d2dca155cfea90b82d27f088d005de7c35a4f5b7ab94325a7463 not found: ID does not exist"
Sep 10 19:39:55.137441 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.137411    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:39:55.142152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.142113    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/auth-test-model-b83810-kserve-796c584795-cjrc2"]
Sep 10 19:39:55.702673 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.702637    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fc455a3-f1f7-4951-8163-05a3e65e1460" path="/var/lib/kubelet/pods/7fc455a3-f1f7-4951-8163-05a3e65e1460/volumes"
Sep 10 19:39:55.703067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:55.703051    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" path="/var/lib/kubelet/pods/a7a986df-8a40-4d87-a8ac-86f5f4233be5/volumes"
Sep 10 19:39:56.035897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:56.035792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:56.036125 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:56.036107    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:56.048588 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:56.048561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:39:56.133948 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:39:56.133906    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:40:06.040921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.040885    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041352    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc455a3-f1f7-4951-8163-05a3e65e1460" containerName="main"
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041366    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc455a3-f1f7-4951-8163-05a3e65e1460" containerName="main"
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041384    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" containerName="main"
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041390    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" containerName="main"
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041481    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fc455a3-f1f7-4951-8163-05a3e65e1460" containerName="main"
Sep 10 19:40:06.041546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.041491    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a986df-8a40-4d87-a8ac-86f5f4233be5" containerName="main"
Sep 10 19:40:06.045847 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.045826    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.049320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.049296    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"sub-test-model-b83810-kserve-self-signed-certs\""
Sep 10 19:40:06.049459 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.049332    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"kube-root-ca.crt\""
Sep 10 19:40:06.049459 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.049288    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"openshift-service-ca.crt\""
Sep 10 19:40:06.049459 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.049293    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"default-dockercfg-vnflj\""
Sep 10 19:40:06.054379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.054356    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:40:06.198091 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.198270 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.198270 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.198270 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.198270 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198201    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.198414 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.198291    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmbpx\" (UniqueName: \"kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.299730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.299730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.299730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmbpx\" (UniqueName: \"kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300055 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.299816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300212 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.300153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300212 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.300201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.300287 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.300239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.302117 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.302097    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.302273 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.302255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.309456 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.309437    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmbpx\" (UniqueName: \"kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx\") pod \"sub-test-model-b83810-kserve-57c4f6f759-828h8\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") " pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.357493 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.357447    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:06.636802 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:06.636768    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:40:06.638134 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:40:06.638109    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1677d1f4_3853_4abb_ad03_77a6b9cf11b6.slice/crio-b651437d909478da19492a5744168e8397f14736c16949fbc06fdd4bc6cc0aff WatchSource:0}: Error finding container b651437d909478da19492a5744168e8397f14736c16949fbc06fdd4bc6cc0aff: Status 404 returned error can't find the container with id b651437d909478da19492a5744168e8397f14736c16949fbc06fdd4bc6cc0aff
Sep 10 19:40:07.187895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:07.187861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" event={"ID":"1677d1f4-3853-4abb-ad03-77a6b9cf11b6","Type":"ContainerStarted","Data":"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"}
Sep 10 19:40:07.187895 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:07.187899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" event={"ID":"1677d1f4-3853-4abb-ad03-77a6b9cf11b6","Type":"ContainerStarted","Data":"b651437d909478da19492a5744168e8397f14736c16949fbc06fdd4bc6cc0aff"}
Sep 10 19:40:07.206593 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:07.206523    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" podStartSLOduration=1.206505107 podStartE2EDuration="1.206505107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:07.205272318 +0000 UTC m=+1466.026328453" watchObservedRunningTime="2026-09-10 19:40:07.206505107 +0000 UTC m=+1466.027561239"
Sep 10 19:40:11.026991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:11.026952    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:40:14.883255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.883223    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:40:14.887240 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.887213    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:14.889522 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.889484    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-846240-serving-cert\""
Sep 10 19:40:14.900203 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.900175    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:40:14.981595 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.981550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:14.981801 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.981671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:14.981801 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.981700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4nmn\" (UniqueName: \"kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:14.981911 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:14.981838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.083048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.083012    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.083048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.083051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4nmn\" (UniqueName: \"kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.083279 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.083098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.083279 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.083152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.083279 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:40:15.083187    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-846240-serving-cert: secret "maas-api-e2e-api-a-846240-serving-cert" not found
Sep 10 19:40:15.083279 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:40:15.083262    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls podName:0d802735-78a6-47a3-8687-12cee06e0fc3 nodeName:}" failed. No retries permitted until 2026-09-10 19:40:15.583240479 +0000 UTC m=+1474.404296607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls") pod "maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" (UID: "0d802735-78a6-47a3-8687-12cee06e0fc3") : secret "maas-api-e2e-api-a-846240-serving-cert" not found
Sep 10 19:40:15.083783 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.083763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.085685 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.085661    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.094220 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.094193    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4nmn\" (UniqueName: \"kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.590433 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.590394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.592940 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.592911    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") pod \"maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:15.802236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:15.802201    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:16.076686 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.076659    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:40:16.078106 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:40:16.078081    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d802735_78a6_47a3_8687_12cee06e0fc3.slice/crio-e5f70b27ac417970370fbcbeae8dc0501e0a42b8cc3d918fb1919f96f76f94bc WatchSource:0}: Error finding container e5f70b27ac417970370fbcbeae8dc0501e0a42b8cc3d918fb1919f96f76f94bc: Status 404 returned error can't find the container with id e5f70b27ac417970370fbcbeae8dc0501e0a42b8cc3d918fb1919f96f76f94bc
Sep 10 19:40:16.079460 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.079437    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:40:16.240902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.240867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" event={"ID":"0d802735-78a6-47a3-8687-12cee06e0fc3","Type":"ContainerStarted","Data":"e5f70b27ac417970370fbcbeae8dc0501e0a42b8cc3d918fb1919f96f76f94bc"}
Sep 10 19:40:16.358440 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.358404    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:16.358726 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.358709    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:16.371133 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:16.371100    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:17.248910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:17.248873    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" event={"ID":"0d802735-78a6-47a3-8687-12cee06e0fc3","Type":"ContainerStarted","Data":"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"}
Sep 10 19:40:17.249390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:17.249128    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:17.260132 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:17.260110    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:40:17.271079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:17.271036    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" podStartSLOduration=2.7897649810000003 podStartE2EDuration="3.271022655s" totalImagesPullingTime="481.257674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:17.268698962 +0000 UTC m=+1476.089755096" watchObservedRunningTime="2026-09-10 19:40:17.271022655 +0000 UTC m=+1476.092078788"
Sep 10 19:40:23.260799 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:23.260764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:40:32.733110 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:32.733073    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:40:41.775409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:41.775382    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:40:41.782337 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:41.782314    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:40:41.782800 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:41.782786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:40:41.789536 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:41.789518    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:40:42.037030 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:42.036956    2567 scope.go:117] "RemoveContainer" containerID="6022f59d41442dd46e2489dfa4d1aaa25635cf945bd3007d991848356d4b5c63"
Sep 10 19:40:42.045861 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:42.045838    2567 scope.go:117] "RemoveContainer" containerID="a90cb984c78fba1f6312f0dc6ca9a45c29599a3be53628630c75353270d78064"
Sep 10 19:40:42.053946 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:42.053916    2567 scope.go:117] "RemoveContainer" containerID="d0bca6a352523088ef35a97ee63034c2beac00e1a0a366eb38ce84f4e145cbd1"
Sep 10 19:40:44.222975 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.222910    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:40:44.233450 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.233415    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.236076 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.236050    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ed39fa-serving-cert\""
Sep 10 19:40:44.240057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.240036    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:40:44.379196 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.379161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.379401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.379212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.379401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.379231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjzjv\" (UniqueName: \"kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.379401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.379291    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.480597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.480495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.480597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.480543    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.480597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.480559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjzjv\" (UniqueName: \"kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.480597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.480581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.481306 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.481281    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.483056 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.483029    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.483223 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.483203    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.491806 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.491780    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjzjv\" (UniqueName: \"kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv\") pod \"maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.547567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.547537    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:44.825016 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:44.824988    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:40:44.827028 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:40:44.827003    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24735939_a957_4061_9aa0_51406dbf5642.slice/crio-447a5e05584de99a0e80a0e82d0748c118443ad9e4ac015451d4c5bba7db59cb WatchSource:0}: Error finding container 447a5e05584de99a0e80a0e82d0748c118443ad9e4ac015451d4c5bba7db59cb: Status 404 returned error can't find the container with id 447a5e05584de99a0e80a0e82d0748c118443ad9e4ac015451d4c5bba7db59cb
Sep 10 19:40:45.421577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:45.421533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" event={"ID":"24735939-a957-4061-9aa0-51406dbf5642","Type":"ContainerStarted","Data":"b9bff6feb0a03eafc7e4d47ce652b945a4fd4e6ce5f9f2bc1047192a246ad21c"}
Sep 10 19:40:45.422053 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:45.421590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" event={"ID":"24735939-a957-4061-9aa0-51406dbf5642","Type":"ContainerStarted","Data":"447a5e05584de99a0e80a0e82d0748c118443ad9e4ac015451d4c5bba7db59cb"}
Sep 10 19:40:45.422053 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:45.421612    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:45.440914 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:45.440859    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" podStartSLOduration=1.138316654 podStartE2EDuration="1.440841152s" totalImagesPullingTime="302.524498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:40:45.439715856 +0000 UTC m=+1504.260771989" watchObservedRunningTime="2026-09-10 19:40:45.440841152 +0000 UTC m=+1504.261897284"
Sep 10 19:40:51.432518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:51.432488    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:40:53.131369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:40:53.131336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:41:08.704527 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.704447    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8063-c0e2f3"]
Sep 10 19:41:08.709295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.709276    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:08.712261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.712224    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bglfh\""
Sep 10 19:41:08.716455 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.716422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-c0e2f3"]
Sep 10 19:41:08.841740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.841701    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lztfs\" (UniqueName: \"kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs\") pod \"test-curl-8063-c0e2f3\" (UID: \"fb0b0884-bf48-4302-9da9-c07538e28d30\") " pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:08.942422 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.942384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lztfs\" (UniqueName: \"kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs\") pod \"test-curl-8063-c0e2f3\" (UID: \"fb0b0884-bf48-4302-9da9-c07538e28d30\") " pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:08.952411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:08.952372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lztfs\" (UniqueName: \"kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs\") pod \"test-curl-8063-c0e2f3\" (UID: \"fb0b0884-bf48-4302-9da9-c07538e28d30\") " pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:09.021391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:09.021300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:09.295130 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:09.295104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-c0e2f3"]
Sep 10 19:41:09.297165 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:41:09.297137    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb0b0884_bf48_4302_9da9_c07538e28d30.slice/crio-ef79517b606d046669096a7e3bb124e5dc149b21162c097b748d259918f1d851 WatchSource:0}: Error finding container ef79517b606d046669096a7e3bb124e5dc149b21162c097b748d259918f1d851: Status 404 returned error can't find the container with id ef79517b606d046669096a7e3bb124e5dc149b21162c097b748d259918f1d851
Sep 10 19:41:09.558858 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:09.558770    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-c0e2f3" event={"ID":"fb0b0884-bf48-4302-9da9-c07538e28d30","Type":"ContainerStarted","Data":"61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2"}
Sep 10 19:41:09.558858 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:09.558816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-c0e2f3" event={"ID":"fb0b0884-bf48-4302-9da9-c07538e28d30","Type":"ContainerStarted","Data":"ef79517b606d046669096a7e3bb124e5dc149b21162c097b748d259918f1d851"}
Sep 10 19:41:09.574902 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:09.574849    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8063-c0e2f3" podStartSLOduration=1.57482772 podStartE2EDuration="1.57482772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:09.574448608 +0000 UTC m=+1528.395504754" watchObservedRunningTime="2026-09-10 19:41:09.57482772 +0000 UTC m=+1528.395883854"
Sep 10 19:41:10.206838 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.206803    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8063-c0e2f3"]
Sep 10 19:41:10.212311 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.212282    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8063-c0e2f3"]
Sep 10 19:41:10.336258 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.336158    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8063-741e5b"]
Sep 10 19:41:10.336623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.336609    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" containerName="test-curl-8063-c0e2f3"
Sep 10 19:41:10.336670 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.336625    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" containerName="test-curl-8063-c0e2f3"
Sep 10 19:41:10.336720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.336711    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" containerName="test-curl-8063-c0e2f3"
Sep 10 19:41:10.339974 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.339958    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:10.347716 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.347695    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-741e5b"]
Sep 10 19:41:10.458026 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.457986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98gf7\" (UniqueName: \"kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7\") pod \"test-curl-8063-741e5b\" (UID: \"ea460859-c3e2-47ca-9a58-f250d2606f1d\") " pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:10.558648 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.558596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gf7\" (UniqueName: \"kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7\") pod \"test-curl-8063-741e5b\" (UID: \"ea460859-c3e2-47ca-9a58-f250d2606f1d\") " pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:10.564235 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:10.564211    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8063-c0e2f3" container="test-curl-8063-c0e2f3"
Sep 10 19:41:10.568041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.568017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98gf7\" (UniqueName: \"kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7\") pod \"test-curl-8063-741e5b\" (UID: \"ea460859-c3e2-47ca-9a58-f250d2606f1d\") " pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:10.650231 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.650202    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:10.928130 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:10.928104    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8063-741e5b"]
Sep 10 19:41:10.930770 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:41:10.930741    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea460859_c3e2_47ca_9a58_f250d2606f1d.slice/crio-90c1673339c373091854fc74ea86d186b2882e816271dcd0874f23731d5751f3 WatchSource:0}: Error finding container 90c1673339c373091854fc74ea86d186b2882e816271dcd0874f23731d5751f3: Status 404 returned error can't find the container with id 90c1673339c373091854fc74ea86d186b2882e816271dcd0874f23731d5751f3
Sep 10 19:41:11.571107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:11.571067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-741e5b" event={"ID":"ea460859-c3e2-47ca-9a58-f250d2606f1d","Type":"ContainerStarted","Data":"0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228"}
Sep 10 19:41:11.571107 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:11.571107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8063-741e5b" event={"ID":"ea460859-c3e2-47ca-9a58-f250d2606f1d","Type":"ContainerStarted","Data":"90c1673339c373091854fc74ea86d186b2882e816271dcd0874f23731d5751f3"}
Sep 10 19:41:11.571614 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:11.571104    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8063-c0e2f3" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" containerName="test-curl-8063-c0e2f3" containerID="cri-o://61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2" gracePeriod=2
Sep 10 19:41:11.586534 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:11.586481    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8063-741e5b" podStartSLOduration=1.586465134 podStartE2EDuration="1.586465134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:11.586124911 +0000 UTC m=+1530.407181044" watchObservedRunningTime="2026-09-10 19:41:11.586465134 +0000 UTC m=+1530.407521303"
Sep 10 19:41:11.588214 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:11.588182    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-c0e2f3\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:11.590032 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:11.590011    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-c0e2f3\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:11.700274 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:11.700233    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-c0e2f3\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:12.185999 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:12.185963    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8063-741e5b"]
Sep 10 19:41:12.191738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:12.191706    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8063-741e5b"]
Sep 10 19:41:12.577314 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:12.577283    2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8063-741e5b" container="test-curl-8063-741e5b"
Sep 10 19:41:13.584580 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:13.584525    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8063-741e5b" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" containerName="test-curl-8063-741e5b" containerID="cri-o://0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228" gracePeriod=2
Sep 10 19:41:13.587093 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:13.587046    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-741e5b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:13.764248 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:13.764223    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:13.890954 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:13.890845    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lztfs\" (UniqueName: \"kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs\") pod \"fb0b0884-bf48-4302-9da9-c07538e28d30\" (UID: \"fb0b0884-bf48-4302-9da9-c07538e28d30\") "
Sep 10 19:41:13.893011 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:13.892978    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs" (OuterVolumeSpecName: "kube-api-access-lztfs") pod "fb0b0884-bf48-4302-9da9-c07538e28d30" (UID: "fb0b0884-bf48-4302-9da9-c07538e28d30"). InnerVolumeSpecName "kube-api-access-lztfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:13.992486 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:13.992448    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lztfs\" (UniqueName: \"kubernetes.io/projected/fb0b0884-bf48-4302-9da9-c07538e28d30-kube-api-access-lztfs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:14.201558 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.201523    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:41:14.592263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.592174    2567 generic.go:356] "Generic (PLEG): container finished" podID="fb0b0884-bf48-4302-9da9-c07538e28d30" containerID="61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2" exitCode=137
Sep 10 19:41:14.592263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.592211    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:14.592263 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.592248    2567 scope.go:117] "RemoveContainer" containerID="61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2"
Sep 10 19:41:14.601721 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.601703    2567 scope.go:117] "RemoveContainer" containerID="61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2"
Sep 10 19:41:14.602006 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:14.601986    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2\": container with ID starting with 61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2 not found: ID does not exist" containerID="61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2"
Sep 10 19:41:14.602077 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:14.602016    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2"} err="failed to get container status \"61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2\": rpc error: code = NotFound desc = could not find container \"61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2\": container with ID starting with 61ded1edbd19fb03eb6a66dd6abad76de03bdb7818d696f97d7d5d3102ce05e2 not found: ID does not exist"
Sep 10 19:41:14.602209 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:14.602191    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-c0e2f3\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:15.271423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:15.271389    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:41:15.302792 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:15.302751    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-c0e2f3\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" pod="opendatahub/test-curl-8063-c0e2f3"
Sep 10 19:41:15.705529 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:15.705497    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb0b0884-bf48-4302-9da9-c07538e28d30" path="/var/lib/kubelet/pods/fb0b0884-bf48-4302-9da9-c07538e28d30/volumes"
Sep 10 19:41:15.766861 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:15.766838    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:15.911261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:15.911226    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98gf7\" (UniqueName: \"kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7\") pod \"ea460859-c3e2-47ca-9a58-f250d2606f1d\" (UID: \"ea460859-c3e2-47ca-9a58-f250d2606f1d\") "
Sep 10 19:41:15.913448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:15.913419    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7" (OuterVolumeSpecName: "kube-api-access-98gf7") pod "ea460859-c3e2-47ca-9a58-f250d2606f1d" (UID: "ea460859-c3e2-47ca-9a58-f250d2606f1d"). InnerVolumeSpecName "kube-api-access-98gf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:16.012608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.012565    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98gf7\" (UniqueName: \"kubernetes.io/projected/ea460859-c3e2-47ca-9a58-f250d2606f1d-kube-api-access-98gf7\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:16.608646 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.608612    2567 generic.go:356] "Generic (PLEG): container finished" podID="ea460859-c3e2-47ca-9a58-f250d2606f1d" containerID="0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228" exitCode=137
Sep 10 19:41:16.608820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.608672    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:16.608820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.608717    2567 scope.go:117] "RemoveContainer" containerID="0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228"
Sep 10 19:41:16.618078 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.618060    2567 scope.go:117] "RemoveContainer" containerID="0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228"
Sep 10 19:41:16.618360 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:16.618343    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228\": container with ID starting with 0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228 not found: ID does not exist" containerID="0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228"
Sep 10 19:41:16.618409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:16.618368    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228"} err="failed to get container status \"0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228\": rpc error: code = NotFound desc = could not find container \"0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228\": container with ID starting with 0447a029e0d1dc3c4045d2b433551a794566e89f9cca08e3a476757b2d2b4228 not found: ID does not exist"
Sep 10 19:41:16.618792 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:16.618767    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8063-741e5b\" is forbidden: User \"system:node:ip-10-0-134-26.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-26.ec2.internal' and this object" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" pod="opendatahub/test-curl-8063-741e5b"
Sep 10 19:41:17.704030 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:17.703997    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" path="/var/lib/kubelet/pods/ea460859-c3e2-47ca-9a58-f250d2606f1d/volumes"
Sep 10 19:41:18.825266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.825237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:41:18.919419 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.919383    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:41:18.919924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.919905    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" containerName="test-curl-8063-741e5b"
Sep 10 19:41:18.919924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.919927    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" containerName="test-curl-8063-741e5b"
Sep 10 19:41:18.920156 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.920050    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea460859-c3e2-47ca-9a58-f250d2606f1d" containerName="test-curl-8063-741e5b"
Sep 10 19:41:18.924653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.924630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:18.926865 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.926843    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-28d250\"/\"rate-test-model-28d250-kserve-self-signed-certs\""
Sep 10 19:41:18.931235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:18.931214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:41:19.042138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042103    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.042138 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.042406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.042406 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x52rq\" (UniqueName: \"kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.042495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.042495 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.042444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.143187 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.143187 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.143435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143219    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.143613 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.143783 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143610    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.144845 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.143241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.144845 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.144817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.145050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.144860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x52rq\" (UniqueName: \"kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.145514 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.145490    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.146720 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.146695    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.146898 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.146877    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.155381 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.155355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x52rq\" (UniqueName: \"kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq\") pod \"rate-test-model-28d250-kserve-b67d6fff5-r66st\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") " pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.234523 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.234485    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:19.510963 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.510909    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:41:19.512511 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:41:19.512485    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac5aaccb_bb44_42c3_8421_6a5fff4cbcc1.slice/crio-0d77acb7ee897fcc6b8c842c228f1d59a6d572ca0c4eddcc48d55d96706fa06c WatchSource:0}: Error finding container 0d77acb7ee897fcc6b8c842c228f1d59a6d572ca0c4eddcc48d55d96706fa06c: Status 404 returned error can't find the container with id 0d77acb7ee897fcc6b8c842c228f1d59a6d572ca0c4eddcc48d55d96706fa06c
Sep 10 19:41:19.629632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.629595    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" event={"ID":"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1","Type":"ContainerStarted","Data":"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"}
Sep 10 19:41:19.629632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.629635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" event={"ID":"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1","Type":"ContainerStarted","Data":"0d77acb7ee897fcc6b8c842c228f1d59a6d572ca0c4eddcc48d55d96706fa06c"}
Sep 10 19:41:19.648897 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:19.648842    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" podStartSLOduration=1.648827832 podStartE2EDuration="1.648827832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:19.646483484 +0000 UTC m=+1538.467539612" watchObservedRunningTime="2026-09-10 19:41:19.648827832 +0000 UTC m=+1538.469883964"
Sep 10 19:41:26.514706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:26.514658    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:41:26.668085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:26.668036    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" podUID="0d802735-78a6-47a3-8687-12cee06e0fc3" containerName="maas-api" containerID="cri-o://a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79" gracePeriod=30
Sep 10 19:41:26.926137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:26.926108    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:41:27.010632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.010593    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca\") pod \"0d802735-78a6-47a3-8687-12cee06e0fc3\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") "
Sep 10 19:41:27.010797 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.010703    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs\") pod \"0d802735-78a6-47a3-8687-12cee06e0fc3\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") "
Sep 10 19:41:27.010797 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.010757    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4nmn\" (UniqueName: \"kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn\") pod \"0d802735-78a6-47a3-8687-12cee06e0fc3\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") "
Sep 10 19:41:27.010886 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.010821    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") pod \"0d802735-78a6-47a3-8687-12cee06e0fc3\" (UID: \"0d802735-78a6-47a3-8687-12cee06e0fc3\") "
Sep 10 19:41:27.010991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.010965    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d802735-78a6-47a3-8687-12cee06e0fc3" (UID: "0d802735-78a6-47a3-8687-12cee06e0fc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:27.011260 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.011230    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d802735-78a6-47a3-8687-12cee06e0fc3-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.012925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.012897    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn" (OuterVolumeSpecName: "kube-api-access-l4nmn") pod "0d802735-78a6-47a3-8687-12cee06e0fc3" (UID: "0d802735-78a6-47a3-8687-12cee06e0fc3"). InnerVolumeSpecName "kube-api-access-l4nmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:27.012925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.012919    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d802735-78a6-47a3-8687-12cee06e0fc3" (UID: "0d802735-78a6-47a3-8687-12cee06e0fc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:27.012925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.012965    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0d802735-78a6-47a3-8687-12cee06e0fc3" (UID: "0d802735-78a6-47a3-8687-12cee06e0fc3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:27.112041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.111915    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4nmn\" (UniqueName: \"kubernetes.io/projected/0d802735-78a6-47a3-8687-12cee06e0fc3-kube-api-access-l4nmn\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.112041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.111983    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.112041 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.111997    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d802735-78a6-47a3-8687-12cee06e0fc3-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:27.675501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.675461    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d802735-78a6-47a3-8687-12cee06e0fc3" containerID="a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79" exitCode=0
Sep 10 19:41:27.675501 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.675507    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"
Sep 10 19:41:27.676067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.675555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" event={"ID":"0d802735-78a6-47a3-8687-12cee06e0fc3","Type":"ContainerDied","Data":"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"}
Sep 10 19:41:27.676067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.675624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v" event={"ID":"0d802735-78a6-47a3-8687-12cee06e0fc3","Type":"ContainerDied","Data":"e5f70b27ac417970370fbcbeae8dc0501e0a42b8cc3d918fb1919f96f76f94bc"}
Sep 10 19:41:27.676067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.675660    2567 scope.go:117] "RemoveContainer" containerID="a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"
Sep 10 19:41:27.686102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.686083    2567 scope.go:117] "RemoveContainer" containerID="a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"
Sep 10 19:41:27.686391 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:27.686374    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79\": container with ID starting with a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79 not found: ID does not exist" containerID="a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"
Sep 10 19:41:27.686470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.686397    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79"} err="failed to get container status \"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79\": rpc error: code = NotFound desc = could not find container \"a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79\": container with ID starting with a984520959b2fe3da2a7d7d9edd9db8967b25cdd1736899aa5e5744c4524ce79 not found: ID does not exist"
Sep 10 19:41:27.703993 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.703970    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:41:27.704835 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:27.704819    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-846240-c5b9d6c66-zhb2v"]
Sep 10 19:41:29.221579 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.221535    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" podUID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" containerName="main" containerID="cri-o://3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14" gracePeriod=45
Sep 10 19:41:29.235297 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.235264    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:29.235435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.235315    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:29.249423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.249397    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:29.480213 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.480141    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:41:29.534987 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.534927    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.534995    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535026    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535046    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535157 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535123    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535217    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzcf9\" (UniqueName: \"kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9\") pod \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\" (UID: \"31b6f70c-a149-4d88-92e2-bbc4d9c755e8\") "
Sep 10 19:41:29.535318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535257    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:29.535417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535315    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home" (OuterVolumeSpecName: "home") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:29.535417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:29.535627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535603    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.535627 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535623    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.535765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.535632    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.537237 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.537211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm" (OuterVolumeSpecName: "dshm") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:29.537372 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.537286    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:29.537471 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.537450    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9" (OuterVolumeSpecName: "kube-api-access-gzcf9") pod "31b6f70c-a149-4d88-92e2-bbc4d9c755e8" (UID: "31b6f70c-a149-4d88-92e2-bbc4d9c755e8"). InnerVolumeSpecName "kube-api-access-gzcf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:29.636547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.636509    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.636547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.636541    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.636547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.636552    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzcf9\" (UniqueName: \"kubernetes.io/projected/31b6f70c-a149-4d88-92e2-bbc4d9c755e8-kube-api-access-gzcf9\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:29.692237 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.692199    2567 generic.go:356] "Generic (PLEG): container finished" podID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" containerID="3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14" exitCode=0
Sep 10 19:41:29.692237 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.692241    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"
Sep 10 19:41:29.692518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.692288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" event={"ID":"31b6f70c-a149-4d88-92e2-bbc4d9c755e8","Type":"ContainerDied","Data":"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"}
Sep 10 19:41:29.692518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.692332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w" event={"ID":"31b6f70c-a149-4d88-92e2-bbc4d9c755e8","Type":"ContainerDied","Data":"ed7c0ce84599403882c9bf391820690906448586eff785e52591caf4e0ee92a5"}
Sep 10 19:41:29.692518 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.692354    2567 scope.go:117] "RemoveContainer" containerID="3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"
Sep 10 19:41:29.703142 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.703126    2567 scope.go:117] "RemoveContainer" containerID="3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"
Sep 10 19:41:29.703247 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.703227    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d802735-78a6-47a3-8687-12cee06e0fc3" path="/var/lib/kubelet/pods/0d802735-78a6-47a3-8687-12cee06e0fc3/volumes"
Sep 10 19:41:29.703421 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:29.703397    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14\": container with ID starting with 3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14 not found: ID does not exist" containerID="3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"
Sep 10 19:41:29.703513 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.703436    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14"} err="failed to get container status \"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14\": rpc error: code = NotFound desc = could not find container \"3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14\": container with ID starting with 3c6bd37a43db8af99770fc14fe521f559a68340e85580d0931c352edf98cbe14 not found: ID does not exist"
Sep 10 19:41:29.704837 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.704812    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:41:29.716382 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.716356    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:41:29.720661 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:29.720637    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/sub-test-model-28d250-kserve-7d8d87567c-42c8w"]
Sep 10 19:41:30.291612 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.291569    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" podUID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" containerName="main" containerID="cri-o://f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863" gracePeriod=45
Sep 10 19:41:30.536193 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.536167    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:41:30.645912 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.645885    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.645924    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.645978    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646015    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmbpx\" (UniqueName: \"kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646048    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646079 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646069    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache\") pod \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\" (UID: \"1677d1f4-3853-4abb-ad03-77a6b9cf11b6\") "
Sep 10 19:41:30.646365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:30.646430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646395    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:30.646430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646407    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:30.646547 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.646520    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home" (OuterVolumeSpecName: "home") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:30.648266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.648237    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm" (OuterVolumeSpecName: "dshm") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:41:30.648266 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.648256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:30.648390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.648272    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx" (OuterVolumeSpecName: "kube-api-access-zmbpx") pod "1677d1f4-3853-4abb-ad03-77a6b9cf11b6" (UID: "1677d1f4-3853-4abb-ad03-77a6b9cf11b6"). InnerVolumeSpecName "kube-api-access-zmbpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:30.699870 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.699842    2567 generic.go:356] "Generic (PLEG): container finished" podID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" containerID="f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863" exitCode=0
Sep 10 19:41:30.700032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.699875    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"
Sep 10 19:41:30.700032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.699926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" event={"ID":"1677d1f4-3853-4abb-ad03-77a6b9cf11b6","Type":"ContainerDied","Data":"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"}
Sep 10 19:41:30.700032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.699975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8" event={"ID":"1677d1f4-3853-4abb-ad03-77a6b9cf11b6","Type":"ContainerDied","Data":"b651437d909478da19492a5744168e8397f14736c16949fbc06fdd4bc6cc0aff"}
Sep 10 19:41:30.700032 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.699998    2567 scope.go:117] "RemoveContainer" containerID="f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"
Sep 10 19:41:30.709831 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.709813    2567 scope.go:117] "RemoveContainer" containerID="f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"
Sep 10 19:41:30.710116 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:41:30.710094    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863\": container with ID starting with f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863 not found: ID does not exist" containerID="f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"
Sep 10 19:41:30.710191 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.710128    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863"} err="failed to get container status \"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863\": rpc error: code = NotFound desc = could not find container \"f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863\": container with ID starting with f5a737d6dc29f11aafb3208e8bc84b13824730770c022684d8ac9f4834357863 not found: ID does not exist"
Sep 10 19:41:30.722657 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.722634    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:41:30.727640 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.727617    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/sub-test-model-b83810-kserve-57c4f6f759-828h8"]
Sep 10 19:41:30.746975 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.746950    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:30.746975 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.746975    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:30.747133 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.746986    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:30.747133 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.746994    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:30.747133 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:30.747003    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmbpx\" (UniqueName: \"kubernetes.io/projected/1677d1f4-3853-4abb-ad03-77a6b9cf11b6-kube-api-access-zmbpx\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:31.705023 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:31.704987    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" path="/var/lib/kubelet/pods/1677d1f4-3853-4abb-ad03-77a6b9cf11b6/volumes"
Sep 10 19:41:31.705439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:31.705424    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" path="/var/lib/kubelet/pods/31b6f70c-a149-4d88-92e2-bbc4d9c755e8/volumes"
Sep 10 19:41:35.526879 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:35.526843    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:41:38.637314 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637272    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637733    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" containerName="main"
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637748    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" containerName="main"
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637761    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" containerName="main"
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637767    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" containerName="main"
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637783    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d802735-78a6-47a3-8687-12cee06e0fc3" containerName="maas-api"
Sep 10 19:41:38.637791 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637788    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d802735-78a6-47a3-8687-12cee06e0fc3" containerName="maas-api"
Sep 10 19:41:38.638124 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637863    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1677d1f4-3853-4abb-ad03-77a6b9cf11b6" containerName="main"
Sep 10 19:41:38.638124 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637871    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b6f70c-a149-4d88-92e2-bbc4d9c755e8" containerName="main"
Sep 10 19:41:38.638124 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.637881    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d802735-78a6-47a3-8687-12cee06e0fc3" containerName="maas-api"
Sep 10 19:41:38.642435 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.642413    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.644715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.644690    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"default-dockercfg-vnflj\""
Sep 10 19:41:38.644862 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.644691    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"kube-root-ca.crt\""
Sep 10 19:41:38.645475 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.645453    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"openshift-service-ca.crt\""
Sep 10 19:41:38.645567 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.645550    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-b83810\"/\"rate-test-model-b83810-kserve-self-signed-certs\""
Sep 10 19:41:38.652405 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.652376    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:41:38.724853 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.724806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.724853 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.724851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.725111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.724879    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.725111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.724905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.725111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.725052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.725111 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.725101    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjtr9\" (UniqueName: \"kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826192    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjtr9\" (UniqueName: \"kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826364 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826583 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826533    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.826632 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.826596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.828560 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.828533    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.828850 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.828832    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.836489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.836459    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjtr9\" (UniqueName: \"kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9\") pod \"rate-test-model-b83810-kserve-5d94c65ccf-q58xr\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") " pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:38.953658 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:38.953607    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:39.231423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:39.231394    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:41:39.233490 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:41:39.233456    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24fe1e7b_7928_420a_bd76_bb4b1573c091.slice/crio-488b94e65a6b3b40bf62468051648710b4ac41a0fff11c8ecdd006b7e66a233a WatchSource:0}: Error finding container 488b94e65a6b3b40bf62468051648710b4ac41a0fff11c8ecdd006b7e66a233a: Status 404 returned error can't find the container with id 488b94e65a6b3b40bf62468051648710b4ac41a0fff11c8ecdd006b7e66a233a
Sep 10 19:41:39.754245 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:39.754209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" event={"ID":"24fe1e7b-7928-420a-bd76-bb4b1573c091","Type":"ContainerStarted","Data":"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"}
Sep 10 19:41:39.754631 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:39.754254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" event={"ID":"24fe1e7b-7928-420a-bd76-bb4b1573c091","Type":"ContainerStarted","Data":"488b94e65a6b3b40bf62468051648710b4ac41a0fff11c8ecdd006b7e66a233a"}
Sep 10 19:41:39.785533 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:39.785478    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" podStartSLOduration=1.785459197 podStartE2EDuration="1.785459197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:41:39.784650219 +0000 UTC m=+1558.605706356" watchObservedRunningTime="2026-09-10 19:41:39.785459197 +0000 UTC m=+1558.606515330"
Sep 10 19:41:48.954798 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:48.954739    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:48.955329 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:48.954812    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:48.967740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:48.967710    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:49.694508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.694469    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:41:49.694800 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.694763    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" podUID="24735939-a957-4061-9aa0-51406dbf5642" containerName="maas-api" containerID="cri-o://b9bff6feb0a03eafc7e4d47ce652b945a4fd4e6ce5f9f2bc1047192a246ad21c" gracePeriod=30
Sep 10 19:41:49.816401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.816366    2567 generic.go:356] "Generic (PLEG): container finished" podID="24735939-a957-4061-9aa0-51406dbf5642" containerID="b9bff6feb0a03eafc7e4d47ce652b945a4fd4e6ce5f9f2bc1047192a246ad21c" exitCode=0
Sep 10 19:41:49.816496 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.816440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" event={"ID":"24735939-a957-4061-9aa0-51406dbf5642","Type":"ContainerDied","Data":"b9bff6feb0a03eafc7e4d47ce652b945a4fd4e6ce5f9f2bc1047192a246ad21c"}
Sep 10 19:41:49.831806 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.831775    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:41:49.967738 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:49.967713    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:41:50.142321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.142284    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca\") pod \"24735939-a957-4061-9aa0-51406dbf5642\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") "
Sep 10 19:41:50.142321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.142326    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls\") pod \"24735939-a957-4061-9aa0-51406dbf5642\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") "
Sep 10 19:41:50.142546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.142454    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs\") pod \"24735939-a957-4061-9aa0-51406dbf5642\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") "
Sep 10 19:41:50.142546 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.142491    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjzjv\" (UniqueName: \"kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv\") pod \"24735939-a957-4061-9aa0-51406dbf5642\" (UID: \"24735939-a957-4061-9aa0-51406dbf5642\") "
Sep 10 19:41:50.142764 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.142737    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24735939-a957-4061-9aa0-51406dbf5642" (UID: "24735939-a957-4061-9aa0-51406dbf5642"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:41:50.144695 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.144663    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv" (OuterVolumeSpecName: "kube-api-access-kjzjv") pod "24735939-a957-4061-9aa0-51406dbf5642" (UID: "24735939-a957-4061-9aa0-51406dbf5642"). InnerVolumeSpecName "kube-api-access-kjzjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:41:50.144825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.144697    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "24735939-a957-4061-9aa0-51406dbf5642" (UID: "24735939-a957-4061-9aa0-51406dbf5642"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:50.144825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.144730    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24735939-a957-4061-9aa0-51406dbf5642" (UID: "24735939-a957-4061-9aa0-51406dbf5642"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:41:50.243997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.243853    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjzjv\" (UniqueName: \"kubernetes.io/projected/24735939-a957-4061-9aa0-51406dbf5642-kube-api-access-kjzjv\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.243997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.243894    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24735939-a957-4061-9aa0-51406dbf5642-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.243997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.243910    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.243997 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.243920    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24735939-a957-4061-9aa0-51406dbf5642-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:41:50.823741 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.823712    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"
Sep 10 19:41:50.823741 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.823704    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz" event={"ID":"24735939-a957-4061-9aa0-51406dbf5642","Type":"ContainerDied","Data":"447a5e05584de99a0e80a0e82d0748c118443ad9e4ac015451d4c5bba7db59cb"}
Sep 10 19:41:50.824064 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.823790    2567 scope.go:117] "RemoveContainer" containerID="b9bff6feb0a03eafc7e4d47ce652b945a4fd4e6ce5f9f2bc1047192a246ad21c"
Sep 10 19:41:50.847390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.847354    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:41:50.852146 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:50.852113    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed39fa-66494f669f-6m8dz"]
Sep 10 19:41:51.706871 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:51.706831    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24735939-a957-4061-9aa0-51406dbf5642" path="/var/lib/kubelet/pods/24735939-a957-4061-9aa0-51406dbf5642/volumes"
Sep 10 19:41:54.227201 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:41:54.227161    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:42:11.428585 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:11.428539    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:42:26.209874 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.209841    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:42:26.210379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.210077    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" podUID="4fb68514-d23f-476b-a1ee-4060987e60fb" containerName="maas-api" containerID="cri-o://368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7" gracePeriod=30
Sep 10 19:42:26.457900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.457869    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:42:26.598689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.598590    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") pod \"4fb68514-d23f-476b-a1ee-4060987e60fb\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") "
Sep 10 19:42:26.598689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.598643    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca\") pod \"4fb68514-d23f-476b-a1ee-4060987e60fb\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") "
Sep 10 19:42:26.598689 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.598669    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhk89\" (UniqueName: \"kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89\") pod \"4fb68514-d23f-476b-a1ee-4060987e60fb\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") "
Sep 10 19:42:26.599021 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.598768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs\") pod \"4fb68514-d23f-476b-a1ee-4060987e60fb\" (UID: \"4fb68514-d23f-476b-a1ee-4060987e60fb\") "
Sep 10 19:42:26.599129 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.599091    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4fb68514-d23f-476b-a1ee-4060987e60fb" (UID: "4fb68514-d23f-476b-a1ee-4060987e60fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:42:26.600853 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.600823    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4fb68514-d23f-476b-a1ee-4060987e60fb" (UID: "4fb68514-d23f-476b-a1ee-4060987e60fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:42:26.601369 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.601345    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4fb68514-d23f-476b-a1ee-4060987e60fb" (UID: "4fb68514-d23f-476b-a1ee-4060987e60fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:42:26.601446 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.601383    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89" (OuterVolumeSpecName: "kube-api-access-vhk89") pod "4fb68514-d23f-476b-a1ee-4060987e60fb" (UID: "4fb68514-d23f-476b-a1ee-4060987e60fb"). InnerVolumeSpecName "kube-api-access-vhk89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:26.699891 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.699855    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:26.699891 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.699886    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb68514-d23f-476b-a1ee-4060987e60fb-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:26.699891 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.699896    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhk89\" (UniqueName: \"kubernetes.io/projected/4fb68514-d23f-476b-a1ee-4060987e60fb-kube-api-access-vhk89\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:26.700150 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:26.699905    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4fb68514-d23f-476b-a1ee-4060987e60fb-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:27.040412 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.040378    2567 generic.go:356] "Generic (PLEG): container finished" podID="4fb68514-d23f-476b-a1ee-4060987e60fb" containerID="368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7" exitCode=0
Sep 10 19:42:27.040608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.040427    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"
Sep 10 19:42:27.040608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.040453    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" event={"ID":"4fb68514-d23f-476b-a1ee-4060987e60fb","Type":"ContainerDied","Data":"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"}
Sep 10 19:42:27.040608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.040496    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk" event={"ID":"4fb68514-d23f-476b-a1ee-4060987e60fb","Type":"ContainerDied","Data":"cee3809999ced56eb7921e38d413d649d57f05fbc38d86dc17bd94570c41018f"}
Sep 10 19:42:27.040608 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.040517    2567 scope.go:117] "RemoveContainer" containerID="368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"
Sep 10 19:42:27.052480 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.052452    2567 scope.go:117] "RemoveContainer" containerID="368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"
Sep 10 19:42:27.052876 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:42:27.052847    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7\": container with ID starting with 368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7 not found: ID does not exist" containerID="368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"
Sep 10 19:42:27.053050 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.052889    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7"} err="failed to get container status \"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7\": rpc error: code = NotFound desc = could not find container \"368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7\": container with ID starting with 368bbec465342c3babf28704bccd19fb3cbfce0e16c63727f8f0a2c878b986a7 not found: ID does not exist"
Sep 10 19:42:27.067093 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.067058    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:42:27.071538 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.071509    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7bfb5d-666f4857f6-7dbbk"]
Sep 10 19:42:27.705338 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:27.705302    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb68514-d23f-476b-a1ee-4060987e60fb" path="/var/lib/kubelet/pods/4fb68514-d23f-476b-a1ee-4060987e60fb/volumes"
Sep 10 19:42:29.336532 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:29.336492    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:42:33.228202 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:33.228168    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:42:34.506900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:34.506803    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:42:42.727875 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:42.727832    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:42:48.248653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.248605    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" podUID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" containerName="main" containerID="cri-o://9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23" gracePeriod=45
Sep 10 19:42:48.504691 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.504624    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:42:48.629192 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629149    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629204    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x52rq\" (UniqueName: \"kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629223    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629274    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629391 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629376    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629402    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm\") pod \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\" (UID: \"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1\") "
Sep 10 19:42:48.629616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629502    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:48.629616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629526    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:48.629765 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629744    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home" (OuterVolumeSpecName: "home") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:48.629823 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629797    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:48.629823 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.629815    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:48.631527 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.631496    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:42:48.631647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.631529    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm" (OuterVolumeSpecName: "dshm") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:48.631647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.631576    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq" (OuterVolumeSpecName: "kube-api-access-x52rq") pod "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" (UID: "ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1"). InnerVolumeSpecName "kube-api-access-x52rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:48.731085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.731046    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:48.731085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.731076    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:48.731085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.731086    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:48.731085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:48.731095    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x52rq\" (UniqueName: \"kubernetes.io/projected/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1-kube-api-access-x52rq\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:49.172647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.172612    2567 generic.go:356] "Generic (PLEG): container finished" podID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" containerID="9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23" exitCode=0
Sep 10 19:42:49.172848 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.172661    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"
Sep 10 19:42:49.172848 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.172658    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" event={"ID":"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1","Type":"ContainerDied","Data":"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"}
Sep 10 19:42:49.172848 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.172712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st" event={"ID":"ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1","Type":"ContainerDied","Data":"0d77acb7ee897fcc6b8c842c228f1d59a6d572ca0c4eddcc48d55d96706fa06c"}
Sep 10 19:42:49.172848 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.172734    2567 scope.go:117] "RemoveContainer" containerID="9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"
Sep 10 19:42:49.182847 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.182828    2567 scope.go:117] "RemoveContainer" containerID="9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"
Sep 10 19:42:49.183183 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:42:49.183162    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23\": container with ID starting with 9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23 not found: ID does not exist" containerID="9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"
Sep 10 19:42:49.183247 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.183192    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23"} err="failed to get container status \"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23\": rpc error: code = NotFound desc = could not find container \"9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23\": container with ID starting with 9d76efe439fc54c5077390a5d0c89f8f14eefd672f22a98a77cd4e58b3e08b23 not found: ID does not exist"
Sep 10 19:42:49.196034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.195999    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:42:49.200512 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.200482    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-28d250/rate-test-model-28d250-kserve-b67d6fff5-r66st"]
Sep 10 19:42:49.526656 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.526556    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" podUID="24fe1e7b-7928-420a-bd76-bb4b1573c091" containerName="main" containerID="cri-o://60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10" gracePeriod=45
Sep 10 19:42:49.711257 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.711215    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" path="/var/lib/kubelet/pods/ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1/volumes"
Sep 10 19:42:49.794872 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.794844    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:42:49.943813 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.943776    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.943813 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.943817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.944097 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.943841    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.944097 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.943880    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjtr9\" (UniqueName: \"kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.944097 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.943957    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.944097 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944053    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs\") pod \"24fe1e7b-7928-420a-bd76-bb4b1573c091\" (UID: \"24fe1e7b-7928-420a-bd76-bb4b1573c091\") "
Sep 10 19:42:49.944319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944147    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:49.944319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944160    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home" (OuterVolumeSpecName: "home") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:49.944319 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944312    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache" (OuterVolumeSpecName: "model-cache") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:49.944508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944405    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:49.944508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944430    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:49.944508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.944468    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:49.946227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.946201    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm" (OuterVolumeSpecName: "dshm") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:42:49.946227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.946212    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:42:49.946227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:49.946226    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9" (OuterVolumeSpecName: "kube-api-access-jjtr9") pod "24fe1e7b-7928-420a-bd76-bb4b1573c091" (UID: "24fe1e7b-7928-420a-bd76-bb4b1573c091"). InnerVolumeSpecName "kube-api-access-jjtr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:42:50.045790 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.045691    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fe1e7b-7928-420a-bd76-bb4b1573c091-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:50.045790 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.045725    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fe1e7b-7928-420a-bd76-bb4b1573c091-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:50.045790 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.045734    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjtr9\" (UniqueName: \"kubernetes.io/projected/24fe1e7b-7928-420a-bd76-bb4b1573c091-kube-api-access-jjtr9\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:42:50.180227 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.180191    2567 generic.go:356] "Generic (PLEG): container finished" podID="24fe1e7b-7928-420a-bd76-bb4b1573c091" containerID="60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10" exitCode=0
Sep 10 19:42:50.180400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.180237    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"
Sep 10 19:42:50.180400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.180272    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" event={"ID":"24fe1e7b-7928-420a-bd76-bb4b1573c091","Type":"ContainerDied","Data":"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"}
Sep 10 19:42:50.180400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.180312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr" event={"ID":"24fe1e7b-7928-420a-bd76-bb4b1573c091","Type":"ContainerDied","Data":"488b94e65a6b3b40bf62468051648710b4ac41a0fff11c8ecdd006b7e66a233a"}
Sep 10 19:42:50.180400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.180328    2567 scope.go:117] "RemoveContainer" containerID="60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"
Sep 10 19:42:50.194166 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.194140    2567 scope.go:117] "RemoveContainer" containerID="60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"
Sep 10 19:42:50.194501 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:42:50.194481    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10\": container with ID starting with 60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10 not found: ID does not exist" containerID="60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"
Sep 10 19:42:50.194597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.194508    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10"} err="failed to get container status \"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10\": rpc error: code = NotFound desc = could not find container \"60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10\": container with ID starting with 60eb66743c20850a9d2b72c9430e48411ec2b33b0bdb7780bd47851fcfb72e10 not found: ID does not exist"
Sep 10 19:42:50.205276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.205243    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:42:50.209955 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:50.209916    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-b83810/rate-test-model-b83810-kserve-5d94c65ccf-q58xr"]
Sep 10 19:42:51.702400 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:42:51.702366    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24fe1e7b-7928-420a-bd76-bb4b1573c091" path="/var/lib/kubelet/pods/24fe1e7b-7928-420a-bd76-bb4b1573c091/volumes"
Sep 10 19:43:09.057246 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057210    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:43:09.057633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057602    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" containerName="main"
Sep 10 19:43:09.057633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057614    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" containerName="main"
Sep 10 19:43:09.057633 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057632    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24fe1e7b-7928-420a-bd76-bb4b1573c091" containerName="main"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057638    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="24fe1e7b-7928-420a-bd76-bb4b1573c091" containerName="main"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057647    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24735939-a957-4061-9aa0-51406dbf5642" containerName="maas-api"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057652    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="24735939-a957-4061-9aa0-51406dbf5642" containerName="maas-api"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057660    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb68514-d23f-476b-a1ee-4060987e60fb" containerName="maas-api"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057665    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb68514-d23f-476b-a1ee-4060987e60fb" containerName="maas-api"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057728    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5aaccb-bb44-42c3-8421-6a5fff4cbcc1" containerName="main"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057736    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="24735939-a957-4061-9aa0-51406dbf5642" containerName="maas-api"
Sep 10 19:43:09.057740 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057742    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb68514-d23f-476b-a1ee-4060987e60fb" containerName="maas-api"
Sep 10 19:43:09.058006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.057747    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="24fe1e7b-7928-420a-bd76-bb4b1573c091" containerName="main"
Sep 10 19:43:09.060900 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.060880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.063122 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.063100    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-544dea-serving-cert\""
Sep 10 19:43:09.073704 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.073678    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:43:09.129320 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.129279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.129509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.129328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.129509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.129450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.129509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.129476    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wcx6\" (UniqueName: \"kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.230485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.230445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.230485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.230597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.230485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.230625    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wcx6\" (UniqueName: \"kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.230485 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.230671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.231474 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.231450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.233137 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.233113    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.233236 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.233217    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.239444 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.239418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wcx6\" (UniqueName: \"kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6\") pod \"maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.374531 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.374437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:09.650959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:09.650919    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:43:09.653195 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:43:09.653167    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63be457e_2270_41b7_a4de_bf9e78f602e9.slice/crio-434db652cae5add060be247d9dfaae06ad2e8495a8609a6041515533d4b6b6b7 WatchSource:0}: Error finding container 434db652cae5add060be247d9dfaae06ad2e8495a8609a6041515533d4b6b6b7: Status 404 returned error can't find the container with id 434db652cae5add060be247d9dfaae06ad2e8495a8609a6041515533d4b6b6b7
Sep 10 19:43:10.293693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:10.293594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" event={"ID":"63be457e-2270-41b7-a4de-bf9e78f602e9","Type":"ContainerStarted","Data":"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"}
Sep 10 19:43:10.293693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:10.293645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" event={"ID":"63be457e-2270-41b7-a4de-bf9e78f602e9","Type":"ContainerStarted","Data":"434db652cae5add060be247d9dfaae06ad2e8495a8609a6041515533d4b6b6b7"}
Sep 10 19:43:10.293693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:10.293678    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:10.311894 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:10.311832    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" podStartSLOduration=0.985772822 podStartE2EDuration="1.311814011s" totalImagesPullingTime="326.041189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:10.309486138 +0000 UTC m=+1649.130542285" watchObservedRunningTime="2026-09-10 19:43:10.311814011 +0000 UTC m=+1649.132870144"
Sep 10 19:43:15.734913 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:15.734863    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:43:16.306384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:16.306355    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:43:48.518163 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.518130    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:43:48.521755 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.521735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.524253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.524226    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-63ed30\"/\"openshift-service-ca.crt\""
Sep 10 19:43:48.524401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.524232    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-63ed30\"/\"default-dockercfg-xjb77\""
Sep 10 19:43:48.524401 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.524277    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-63ed30\"/\"ipp-route-63ed30-kserve-self-signed-certs\""
Sep 10 19:43:48.524819 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.524805    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-63ed30\"/\"kube-root-ca.crt\""
Sep 10 19:43:48.530665 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.530640    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:43:48.699298 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699265    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.699509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l6d9\" (UniqueName: \"kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.699509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699416    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.699509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.699509 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.699679 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.699511    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.800959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.800835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l6d9\" (UniqueName: \"kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.800959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.800888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.800959 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.800927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801251 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.800985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801251 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.801006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801251 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.801078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.801375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801413 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.801392    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.801709 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.801688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.803291 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.803270    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.803586 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.803568    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.810827 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.810794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l6d9\" (UniqueName: \"kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9\") pod \"ipp-route-63ed30-kserve-5c4f67f7b7-zw655\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") " pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:48.832325 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:48.832274    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:49.110484 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:49.110458    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:43:49.112818 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:43:49.112787    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1585324b_d07f_410b_b971_fe9384a1724d.slice/crio-eff1c87ba65b8eed3dee3a3cfdf9537a79ba2de89228874f330828c0298bd02f WatchSource:0}: Error finding container eff1c87ba65b8eed3dee3a3cfdf9537a79ba2de89228874f330828c0298bd02f: Status 404 returned error can't find the container with id eff1c87ba65b8eed3dee3a3cfdf9537a79ba2de89228874f330828c0298bd02f
Sep 10 19:43:49.517694 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:49.517655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" event={"ID":"1585324b-d07f-410b-b971-fe9384a1724d","Type":"ContainerStarted","Data":"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"}
Sep 10 19:43:49.517858 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:49.517705    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" event={"ID":"1585324b-d07f-410b-b971-fe9384a1724d","Type":"ContainerStarted","Data":"eff1c87ba65b8eed3dee3a3cfdf9537a79ba2de89228874f330828c0298bd02f"}
Sep 10 19:43:49.538315 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:49.538256    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" podStartSLOduration=1.538237617 podStartE2EDuration="1.538237617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:43:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:43:49.536467735 +0000 UTC m=+1688.357523870" watchObservedRunningTime="2026-09-10 19:43:49.538237617 +0000 UTC m=+1688.359293750"
Sep 10 19:43:52.523655 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:52.523612    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:43:58.833104 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:58.833070    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:58.833520 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:58.833122    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:58.845903 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:58.845875    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:43:59.584243 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:43:59.584213    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:44:12.822569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:12.822481    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:44:24.685473 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.685437    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:24.689471 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.689445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.691851 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.691823    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-07ffe4-serving-cert\""
Sep 10 19:44:24.706647 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.706619    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:24.858345 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.858304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.858526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.858361    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n6dr\" (UniqueName: \"kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.858526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.858385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.858526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.858418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.959286 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.959181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.959286 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.959247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n6dr\" (UniqueName: \"kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.959286 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.959279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.959604 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.959306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.959983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.959961    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.961805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.961783    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.961912 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.961869    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:24.967983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:24.967954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n6dr\" (UniqueName: \"kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr\") pod \"maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:25.001005 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.000969    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:25.277728 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.277700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:25.279461 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:44:25.279432    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f03e35_f5f0_4574_9a37_877613dbb6c4.slice/crio-1fcb9a30421b73ee65d71ae93769e912a8e33d2355cbb264adc595cbfd0bb6e7 WatchSource:0}: Error finding container 1fcb9a30421b73ee65d71ae93769e912a8e33d2355cbb264adc595cbfd0bb6e7: Status 404 returned error can't find the container with id 1fcb9a30421b73ee65d71ae93769e912a8e33d2355cbb264adc595cbfd0bb6e7
Sep 10 19:44:25.721178 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.721143    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" event={"ID":"97f03e35-f5f0-4574-9a37-877613dbb6c4","Type":"ContainerStarted","Data":"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"}
Sep 10 19:44:25.721573 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.721197    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" event={"ID":"97f03e35-f5f0-4574-9a37-877613dbb6c4","Type":"ContainerStarted","Data":"1fcb9a30421b73ee65d71ae93769e912a8e33d2355cbb264adc595cbfd0bb6e7"}
Sep 10 19:44:25.721573 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.721220    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:25.739194 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:25.739134    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" podStartSLOduration=1.4160121110000001 podStartE2EDuration="1.739114904s" totalImagesPullingTime="323.102793ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:44:25.736896791 +0000 UTC m=+1724.557952925" watchObservedRunningTime="2026-09-10 19:44:25.739114904 +0000 UTC m=+1724.560171037"
Sep 10 19:44:31.733160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:31.733123    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:33.036317 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:33.036271    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:44:55.281118 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.281084    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:55.281691 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.281286    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" podUID="97f03e35-f5f0-4574-9a37-877613dbb6c4" containerName="maas-api" containerID="cri-o://fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3" gracePeriod=30
Sep 10 19:44:55.528914 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.528883    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:55.656983 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.656926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca\") pod \"97f03e35-f5f0-4574-9a37-877613dbb6c4\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") "
Sep 10 19:44:55.657235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.657005    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls\") pod \"97f03e35-f5f0-4574-9a37-877613dbb6c4\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") "
Sep 10 19:44:55.657235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.657039    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n6dr\" (UniqueName: \"kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr\") pod \"97f03e35-f5f0-4574-9a37-877613dbb6c4\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") "
Sep 10 19:44:55.657235 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.657057    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs\") pod \"97f03e35-f5f0-4574-9a37-877613dbb6c4\" (UID: \"97f03e35-f5f0-4574-9a37-877613dbb6c4\") "
Sep 10 19:44:55.657428 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.657328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "97f03e35-f5f0-4574-9a37-877613dbb6c4" (UID: "97f03e35-f5f0-4574-9a37-877613dbb6c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:44:55.659344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.659311    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "97f03e35-f5f0-4574-9a37-877613dbb6c4" (UID: "97f03e35-f5f0-4574-9a37-877613dbb6c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:55.659462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.659394    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "97f03e35-f5f0-4574-9a37-877613dbb6c4" (UID: "97f03e35-f5f0-4574-9a37-877613dbb6c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:44:55.659462 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.659412    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr" (OuterVolumeSpecName: "kube-api-access-8n6dr") pod "97f03e35-f5f0-4574-9a37-877613dbb6c4" (UID: "97f03e35-f5f0-4574-9a37-877613dbb6c4"). InnerVolumeSpecName "kube-api-access-8n6dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:44:55.758869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.758833    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97f03e35-f5f0-4574-9a37-877613dbb6c4-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.758869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.758864    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.758869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.758873    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n6dr\" (UniqueName: \"kubernetes.io/projected/97f03e35-f5f0-4574-9a37-877613dbb6c4-kube-api-access-8n6dr\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.758869 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.758884    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f03e35-f5f0-4574-9a37-877613dbb6c4-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:44:55.896782 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.896750    2567 generic.go:356] "Generic (PLEG): container finished" podID="97f03e35-f5f0-4574-9a37-877613dbb6c4" containerID="fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3" exitCode=0
Sep 10 19:44:55.896960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.896795    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" event={"ID":"97f03e35-f5f0-4574-9a37-877613dbb6c4","Type":"ContainerDied","Data":"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"}
Sep 10 19:44:55.896960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.896833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn" event={"ID":"97f03e35-f5f0-4574-9a37-877613dbb6c4","Type":"ContainerDied","Data":"1fcb9a30421b73ee65d71ae93769e912a8e33d2355cbb264adc595cbfd0bb6e7"}
Sep 10 19:44:55.896960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.896854    2567 scope.go:117] "RemoveContainer" containerID="fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"
Sep 10 19:44:55.896960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.896798    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"
Sep 10 19:44:55.906599 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.906569    2567 scope.go:117] "RemoveContainer" containerID="fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"
Sep 10 19:44:55.906964 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:44:55.906918    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3\": container with ID starting with fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3 not found: ID does not exist" containerID="fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"
Sep 10 19:44:55.907066 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.906973    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3"} err="failed to get container status \"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3\": rpc error: code = NotFound desc = could not find container \"fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3\": container with ID starting with fcb975b351e942593404be4441b7471442aa77ab49ffbfa253820011b72903b3 not found: ID does not exist"
Sep 10 19:44:55.918121 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.918076    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:55.923569 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:55.923527    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-07ffe4-b66dc7f54-k2zfn"]
Sep 10 19:44:57.703774 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:44:57.703739    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97f03e35-f5f0-4574-9a37-877613dbb6c4" path="/var/lib/kubelet/pods/97f03e35-f5f0-4574-9a37-877613dbb6c4/volumes"
Sep 10 19:45:00.026080 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.026040    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:45:00.141289 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.141249    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"]
Sep 10 19:45:00.141782 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.141761    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97f03e35-f5f0-4574-9a37-877613dbb6c4" containerName="maas-api"
Sep 10 19:45:00.141894 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.141785    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="97f03e35-f5f0-4574-9a37-877613dbb6c4" containerName="maas-api"
Sep 10 19:45:00.141981 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.141891    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="97f03e35-f5f0-4574-9a37-877613dbb6c4" containerName="maas-api"
Sep 10 19:45:00.146510 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.146481    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"]
Sep 10 19:45:00.146653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.146593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"
Sep 10 19:45:00.148783 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.148756    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-tjmqn\""
Sep 10 19:45:00.150405 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.150382    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"
Sep 10 19:45:00.153206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.153180    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"]
Sep 10 19:45:00.157033 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.156861    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"]
Sep 10 19:45:00.163173 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.163149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"
Sep 10 19:45:00.173039 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.173012    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"
Sep 10 19:45:00.442663 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.442637    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"]
Sep 10 19:45:00.444450 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:45:00.444424    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a843de_3f7c_49b8_ba96_7e6444eeda56.slice/crio-dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6 WatchSource:0}: Error finding container dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6: Status 404 returned error can't find the container with id dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6
Sep 10 19:45:00.463018 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.462136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"]
Sep 10 19:45:00.466693 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:45:00.466659    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ffbc37_8b4b_44f3_bf8a_12c115532257.slice/crio-22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf WatchSource:0}: Error finding container 22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf: Status 404 returned error can't find the container with id 22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf
Sep 10 19:45:00.927727 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.927690    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q" event={"ID":"b9ffbc37-8b4b-44f3-bf8a-12c115532257","Type":"ContainerStarted","Data":"0de880c7d9a59467e921fa7ec44137bdde42f712ee030ceb887bd1cb62ba94e0"}
Sep 10 19:45:00.927970 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.927756    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q" event={"ID":"b9ffbc37-8b4b-44f3-bf8a-12c115532257","Type":"ContainerStarted","Data":"22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf"}
Sep 10 19:45:00.929451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.929422    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m" event={"ID":"93a843de-3f7c-49b8-ba96-7e6444eeda56","Type":"ContainerStarted","Data":"9e44f587f2b7555f459f231a6d5636d2272979ad9e5f039799c4e0cb7f51322f"}
Sep 10 19:45:00.929553 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.929466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m" event={"ID":"93a843de-3f7c-49b8-ba96-7e6444eeda56","Type":"ContainerStarted","Data":"dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6"}
Sep 10 19:45:00.945697 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.945644    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q" podStartSLOduration=0.945627397 podStartE2EDuration="945.627397ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:00.944788216 +0000 UTC m=+1759.765844350" watchObservedRunningTime="2026-09-10 19:45:00.945627397 +0000 UTC m=+1759.766683529"
Sep 10 19:45:00.960759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:00.960687    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m" podStartSLOduration=0.96066722 podStartE2EDuration="960.66722ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:45:00.959963829 +0000 UTC m=+1759.781019962" watchObservedRunningTime="2026-09-10 19:45:00.96066722 +0000 UTC m=+1759.781723354"
Sep 10 19:45:01.937288 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:01.937248    2567 generic.go:356] "Generic (PLEG): container finished" podID="93a843de-3f7c-49b8-ba96-7e6444eeda56" containerID="9e44f587f2b7555f459f231a6d5636d2272979ad9e5f039799c4e0cb7f51322f" exitCode=0
Sep 10 19:45:01.937722 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:01.937331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m" event={"ID":"93a843de-3f7c-49b8-ba96-7e6444eeda56","Type":"ContainerDied","Data":"9e44f587f2b7555f459f231a6d5636d2272979ad9e5f039799c4e0cb7f51322f"}
Sep 10 19:45:01.938957 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:01.938917    2567 generic.go:356] "Generic (PLEG): container finished" podID="b9ffbc37-8b4b-44f3-bf8a-12c115532257" containerID="0de880c7d9a59467e921fa7ec44137bdde42f712ee030ceb887bd1cb62ba94e0" exitCode=0
Sep 10 19:45:01.939090 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:01.938971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q" event={"ID":"b9ffbc37-8b4b-44f3-bf8a-12c115532257","Type":"ContainerDied","Data":"0de880c7d9a59467e921fa7ec44137bdde42f712ee030ceb887bd1cb62ba94e0"}
Sep 10 19:45:03.085057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.085026    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"
Sep 10 19:45:03.118255 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.118225    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"
Sep 10 19:45:03.952592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.952558    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"
Sep 10 19:45:03.952592 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.952583    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m" event={"ID":"93a843de-3f7c-49b8-ba96-7e6444eeda56","Type":"ContainerDied","Data":"dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6"}
Sep 10 19:45:03.952814 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.952622    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dad45b3b32fb6f2fbed2b21e7c4ebcaa22dcf13a18175172d76864aeb0174bf6"
Sep 10 19:45:03.954048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.954029    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"
Sep 10 19:45:03.954048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.954033    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q" event={"ID":"b9ffbc37-8b4b-44f3-bf8a-12c115532257","Type":"ContainerDied","Data":"22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf"}
Sep 10 19:45:03.954215 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:03.954067    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22d9820a83548bc7da54d56676ca5e60df49ccf1773981c8befc1d18faa270cf"
Sep 10 19:45:19.520443 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:19.520395    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:45:27.466950 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:27.466898    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:45:33.229515 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:33.229479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:45:41.819379 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:41.819347    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:45:41.827071 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:41.827040    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:45:41.829328 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:41.829303    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:45:41.836489 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:41.836455    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:45:42.491361 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.491271    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" podUID="1585324b-d07f-410b-b971-fe9384a1724d" containerName="main" containerID="cri-o://b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1" gracePeriod=45
Sep 10 19:45:42.749365 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.749288    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:45:42.916910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.916868    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.916910 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.916914    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.916959    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l6d9\" (UniqueName: \"kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.916985    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917014    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917100    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir\") pod \"1585324b-d07f-410b-b971-fe9384a1724d\" (UID: \"1585324b-d07f-410b-b971-fe9384a1724d\") "
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917266    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache" (OuterVolumeSpecName: "model-cache") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917380    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home" (OuterVolumeSpecName: "home") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917428    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:42.917439 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.917438    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:42.919449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.919425    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:45:42.919449 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.919439    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm" (OuterVolumeSpecName: "dshm") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:45:42.919617 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:42.919447    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9" (OuterVolumeSpecName: "kube-api-access-6l6d9") pod "1585324b-d07f-410b-b971-fe9384a1724d" (UID: "1585324b-d07f-410b-b971-fe9384a1724d"). InnerVolumeSpecName "kube-api-access-6l6d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:45:43.018926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.018827    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:43.018926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.018861    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1585324b-d07f-410b-b971-fe9384a1724d-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:43.018926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.018873    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:43.018926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.018882    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l6d9\" (UniqueName: \"kubernetes.io/projected/1585324b-d07f-410b-b971-fe9384a1724d-kube-api-access-6l6d9\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:43.018926 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.018893    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1585324b-d07f-410b-b971-fe9384a1724d-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:45:43.180279 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.180248    2567 generic.go:356] "Generic (PLEG): container finished" podID="1585324b-d07f-410b-b971-fe9384a1724d" containerID="b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1" exitCode=0
Sep 10 19:45:43.180452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.180291    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"
Sep 10 19:45:43.180452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.180303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" event={"ID":"1585324b-d07f-410b-b971-fe9384a1724d","Type":"ContainerDied","Data":"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"}
Sep 10 19:45:43.180452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.180335    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655" event={"ID":"1585324b-d07f-410b-b971-fe9384a1724d","Type":"ContainerDied","Data":"eff1c87ba65b8eed3dee3a3cfdf9537a79ba2de89228874f330828c0298bd02f"}
Sep 10 19:45:43.180452 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.180353    2567 scope.go:117] "RemoveContainer" containerID="b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"
Sep 10 19:45:43.198536 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.198511    2567 scope.go:117] "RemoveContainer" containerID="b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"
Sep 10 19:45:43.198869 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:45:43.198845    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1\": container with ID starting with b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1 not found: ID does not exist" containerID="b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"
Sep 10 19:45:43.199006 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.198875    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1"} err="failed to get container status \"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1\": rpc error: code = NotFound desc = could not find container \"b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1\": container with ID starting with b94f80eaf8c11e6d8ca27c2f249026f6748ce5d738c534e40eb4444ba99b8dd1 not found: ID does not exist"
Sep 10 19:45:43.210270 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.210239    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:45:43.217292 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.217263    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-63ed30/ipp-route-63ed30-kserve-5c4f67f7b7-zw655"]
Sep 10 19:45:43.702820 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:45:43.702780    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1585324b-d07f-410b-b971-fe9384a1724d" path="/var/lib/kubelet/pods/1585324b-d07f-410b-b971-fe9384a1724d/volumes"
Sep 10 19:46:22.594825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.594789    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:46:22.595348 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.595034    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" podUID="63be457e-2270-41b7-a4de-bf9e78f602e9" containerName="maas-api" containerID="cri-o://668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1" gracePeriod=30
Sep 10 19:46:22.853261 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.853186    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:46:22.997321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.997285    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls\") pod \"63be457e-2270-41b7-a4de-bf9e78f602e9\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") "
Sep 10 19:46:22.997493 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.997346    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs\") pod \"63be457e-2270-41b7-a4de-bf9e78f602e9\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") "
Sep 10 19:46:22.997493 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.997389    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wcx6\" (UniqueName: \"kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6\") pod \"63be457e-2270-41b7-a4de-bf9e78f602e9\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") "
Sep 10 19:46:22.997493 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.997425    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca\") pod \"63be457e-2270-41b7-a4de-bf9e78f602e9\" (UID: \"63be457e-2270-41b7-a4de-bf9e78f602e9\") "
Sep 10 19:46:22.997859 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.997835    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63be457e-2270-41b7-a4de-bf9e78f602e9" (UID: "63be457e-2270-41b7-a4de-bf9e78f602e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:22.999616 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.999589    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "63be457e-2270-41b7-a4de-bf9e78f602e9" (UID: "63be457e-2270-41b7-a4de-bf9e78f602e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:22.999737 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.999673    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63be457e-2270-41b7-a4de-bf9e78f602e9" (UID: "63be457e-2270-41b7-a4de-bf9e78f602e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:22.999737 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:22.999707    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6" (OuterVolumeSpecName: "kube-api-access-6wcx6") pod "63be457e-2270-41b7-a4de-bf9e78f602e9" (UID: "63be457e-2270-41b7-a4de-bf9e78f602e9"). InnerVolumeSpecName "kube-api-access-6wcx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:23.098697 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.098653    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:23.098697 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.098690    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63be457e-2270-41b7-a4de-bf9e78f602e9-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:23.098697 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.098699    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wcx6\" (UniqueName: \"kubernetes.io/projected/63be457e-2270-41b7-a4de-bf9e78f602e9-kube-api-access-6wcx6\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:23.098964 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.098708    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63be457e-2270-41b7-a4de-bf9e78f602e9-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:23.419921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.419881    2567 generic.go:356] "Generic (PLEG): container finished" podID="63be457e-2270-41b7-a4de-bf9e78f602e9" containerID="668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1" exitCode=0
Sep 10 19:46:23.420128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.419925    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"
Sep 10 19:46:23.420128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.419925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" event={"ID":"63be457e-2270-41b7-a4de-bf9e78f602e9","Type":"ContainerDied","Data":"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"}
Sep 10 19:46:23.420128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.420060    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl" event={"ID":"63be457e-2270-41b7-a4de-bf9e78f602e9","Type":"ContainerDied","Data":"434db652cae5add060be247d9dfaae06ad2e8495a8609a6041515533d4b6b6b7"}
Sep 10 19:46:23.420128 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.420087    2567 scope.go:117] "RemoveContainer" containerID="668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"
Sep 10 19:46:23.430324 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.430301    2567 scope.go:117] "RemoveContainer" containerID="668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"
Sep 10 19:46:23.430719 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:46:23.430697    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1\": container with ID starting with 668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1 not found: ID does not exist" containerID="668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"
Sep 10 19:46:23.430766 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.430730    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1"} err="failed to get container status \"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1\": rpc error: code = NotFound desc = could not find container \"668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1\": container with ID starting with 668bd43373e6e863666ca2918a43e0afe6072714e440f01a287886bcf11fa6a1 not found: ID does not exist"
Sep 10 19:46:23.442429 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.442389    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:46:23.447115 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.447083    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-544dea-bf4488c49-p4fjl"]
Sep 10 19:46:23.703386 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:23.703294    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63be457e-2270-41b7-a4de-bf9e78f602e9" path="/var/lib/kubelet/pods/63be457e-2270-41b7-a4de-bf9e78f602e9/volumes"
Sep 10 19:46:26.132834 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:26.132795    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:46:40.538030 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:40.537992    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"]
Sep 10 19:46:40.540208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:40.540180    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-28d250-29817825-fnq2q"]
Sep 10 19:46:41.702899 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:41.702866    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ffbc37-8b4b-44f3-bf8a-12c115532257" path="/var/lib/kubelet/pods/b9ffbc37-8b4b-44f3-bf8a-12c115532257/volumes"
Sep 10 19:46:43.622105 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:43.622066    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:46:58.880843 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:58.880806    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:46:58.881490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:58.881053    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" podUID="78391b61-ba56-4006-8989-c3d620be146e" containerName="maas-api" containerID="cri-o://289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e" gracePeriod=30
Sep 10 19:46:58.921731 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:58.921692    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"]
Sep 10 19:46:58.924405 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:58.924373    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b83810-29817825-9hz7m"]
Sep 10 19:46:59.141545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.141472    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:46:59.230049 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.230006    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs\") pod \"78391b61-ba56-4006-8989-c3d620be146e\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") "
Sep 10 19:46:59.230256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.230095    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhv52\" (UniqueName: \"kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52\") pod \"78391b61-ba56-4006-8989-c3d620be146e\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") "
Sep 10 19:46:59.230256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.230231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") pod \"78391b61-ba56-4006-8989-c3d620be146e\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") "
Sep 10 19:46:59.230374 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.230284    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca\") pod \"78391b61-ba56-4006-8989-c3d620be146e\" (UID: \"78391b61-ba56-4006-8989-c3d620be146e\") "
Sep 10 19:46:59.230703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.230674    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78391b61-ba56-4006-8989-c3d620be146e" (UID: "78391b61-ba56-4006-8989-c3d620be146e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 19:46:59.232284 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.232256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "78391b61-ba56-4006-8989-c3d620be146e" (UID: "78391b61-ba56-4006-8989-c3d620be146e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:59.232404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.232356    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78391b61-ba56-4006-8989-c3d620be146e" (UID: "78391b61-ba56-4006-8989-c3d620be146e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:46:59.232461 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.232402    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52" (OuterVolumeSpecName: "kube-api-access-rhv52") pod "78391b61-ba56-4006-8989-c3d620be146e" (UID: "78391b61-ba56-4006-8989-c3d620be146e"). InnerVolumeSpecName "kube-api-access-rhv52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:46:59.331732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.331688    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-maas-api-tls\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.331732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.331721    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78391b61-ba56-4006-8989-c3d620be146e-openshift-service-ca\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.331732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.331732    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78391b61-ba56-4006-8989-c3d620be146e-metrics-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.331732 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.331741    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhv52\" (UniqueName: \"kubernetes.io/projected/78391b61-ba56-4006-8989-c3d620be146e-kube-api-access-rhv52\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:46:59.631706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.631669    2567 generic.go:356] "Generic (PLEG): container finished" podID="78391b61-ba56-4006-8989-c3d620be146e" containerID="289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e" exitCode=0
Sep 10 19:46:59.631924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.631716    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"
Sep 10 19:46:59.631924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.631761    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" event={"ID":"78391b61-ba56-4006-8989-c3d620be146e","Type":"ContainerDied","Data":"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"}
Sep 10 19:46:59.631924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.631822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp" event={"ID":"78391b61-ba56-4006-8989-c3d620be146e","Type":"ContainerDied","Data":"4d56f50cdac51bb3c91d5db911788c89af5ff0a6c6e260e0b52336e323652532"}
Sep 10 19:46:59.631924 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.631852    2567 scope.go:117] "RemoveContainer" containerID="289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"
Sep 10 19:46:59.642048 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.642025    2567 scope.go:117] "RemoveContainer" containerID="289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"
Sep 10 19:46:59.642361 ip-10-0-134-26 kubenswrapper[2567]: E0910 19:46:59.642343    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e\": container with ID starting with 289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e not found: ID does not exist" containerID="289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"
Sep 10 19:46:59.642424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.642375    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e"} err="failed to get container status \"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e\": rpc error: code = NotFound desc = could not find container \"289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e\": container with ID starting with 289099de0deed7fbbb1d08545f9c8c951d82195d34e695988c0863ee4108a34e not found: ID does not exist"
Sep 10 19:46:59.655321 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.655284    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:46:59.659823 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.659794    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b83810-6bd498d69f-vl4bp"]
Sep 10 19:46:59.703825 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.703785    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78391b61-ba56-4006-8989-c3d620be146e" path="/var/lib/kubelet/pods/78391b61-ba56-4006-8989-c3d620be146e/volumes"
Sep 10 19:46:59.704171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:46:59.704155    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93a843de-3f7c-49b8-ba96-7e6444eeda56" path="/var/lib/kubelet/pods/93a843de-3f7c-49b8-ba96-7e6444eeda56/volumes"
Sep 10 19:47:01.531242 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:47:01.531201    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:47:10.528225 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:47:10.528126    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:48:33.428748 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:48:33.428710    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:49:01.638963 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:49:01.638857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:49:09.427145 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:49:09.427106    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:49:17.727247 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:49:17.727203    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:49:37.228703 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:49:37.228659    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:49:43.126378 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:49:43.126339    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:50:41.859112 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:50:41.859079    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:50:41.866291 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:50:41.866263    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:50:41.870119 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:50:41.870095    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:50:41.877076 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:50:41.877051    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r7vb4_0154061b-3847-436f-87bc-7fe4c8e4c399/ovn-acl-logging/0.log"
Sep 10 19:51:09.927262 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:09.927223    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:51:18.120833 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:18.120792    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:51:40.334524 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334436    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334883    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78391b61-ba56-4006-8989-c3d620be146e" containerName="maas-api"
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334896    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="78391b61-ba56-4006-8989-c3d620be146e" containerName="maas-api"
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334911    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ffbc37-8b4b-44f3-bf8a-12c115532257" containerName="cleanup"
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334917    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ffbc37-8b4b-44f3-bf8a-12c115532257" containerName="cleanup"
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334928    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93a843de-3f7c-49b8-ba96-7e6444eeda56" containerName="cleanup"
Sep 10 19:51:40.334960 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334951    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="93a843de-3f7c-49b8-ba96-7e6444eeda56" containerName="cleanup"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334975    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63be457e-2270-41b7-a4de-bf9e78f602e9" containerName="maas-api"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334980    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="63be457e-2270-41b7-a4de-bf9e78f602e9" containerName="maas-api"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334986    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1585324b-d07f-410b-b971-fe9384a1724d" containerName="main"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.334992    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1585324b-d07f-410b-b971-fe9384a1724d" containerName="main"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.335056    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="63be457e-2270-41b7-a4de-bf9e78f602e9" containerName="maas-api"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.335064    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ffbc37-8b4b-44f3-bf8a-12c115532257" containerName="cleanup"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.335072    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="93a843de-3f7c-49b8-ba96-7e6444eeda56" containerName="cleanup"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.335077    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1585324b-d07f-410b-b971-fe9384a1724d" containerName="main"
Sep 10 19:51:40.335171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.335084    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="78391b61-ba56-4006-8989-c3d620be146e" containerName="maas-api"
Sep 10 19:51:40.338102 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.338085    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.340404 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.340384    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-2517a652-kserve-self-signed-certs\""
Sep 10 19:51:40.348570 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.348545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:51:40.430203 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430161    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.430203 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.430448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.430448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.430448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsfw9\" (UniqueName: \"kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.430448 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.430385    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.530911 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.530868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.530911 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.530908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.530980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531024    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsfw9\" (UniqueName: \"kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531208 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531459 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531414    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.531526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.531436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.533383 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.533355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.533651 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.533631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.539673 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.539649    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsfw9\" (UniqueName: \"kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9\") pod \"e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") " pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.648203 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.648168    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:40.924730 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:40.924699    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:51:40.926373 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:51:40.926349    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4886654d_28d6_4d39_b9fd_45596b6015f5.slice/crio-dfa136a464cf318d38c48383a63237c28b57493b1ea2a74e63df9b598cdc7665 WatchSource:0}: Error finding container dfa136a464cf318d38c48383a63237c28b57493b1ea2a74e63df9b598cdc7665: Status 404 returned error can't find the container with id dfa136a464cf318d38c48383a63237c28b57493b1ea2a74e63df9b598cdc7665
Sep 10 19:51:41.224787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:41.224694    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" event={"ID":"4886654d-28d6-4d39-b9fd-45596b6015f5","Type":"ContainerStarted","Data":"c857580555f1ea5e848eb15c3fce968d39cc79f0bab7d2dbf7edc322562778a8"}
Sep 10 19:51:41.224787 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:41.224736    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" event={"ID":"4886654d-28d6-4d39-b9fd-45596b6015f5","Type":"ContainerStarted","Data":"dfa136a464cf318d38c48383a63237c28b57493b1ea2a74e63df9b598cdc7665"}
Sep 10 19:51:41.243508 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:41.243454    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" podStartSLOduration=1.243434996 podStartE2EDuration="1.243434996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:51:41.242176445 +0000 UTC m=+2160.063232579" watchObservedRunningTime="2026-09-10 19:51:41.243434996 +0000 UTC m=+2160.064491129"
Sep 10 19:51:42.242543 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:42.242510    2567 scope.go:117] "RemoveContainer" containerID="9e44f587f2b7555f459f231a6d5636d2272979ad9e5f039799c4e0cb7f51322f"
Sep 10 19:51:42.251598 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:42.251573    2567 scope.go:117] "RemoveContainer" containerID="0de880c7d9a59467e921fa7ec44137bdde42f712ee030ceb887bd1cb62ba94e0"
Sep 10 19:51:42.834005 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:42.833964    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:51:50.648775 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:50.648730    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:50.649256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:50.648801    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:50.661529 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:50.661495    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:51.292795 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:51.292759    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:51:55.929135 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:51:55.929094    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:52:29.402785 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:29.402750    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:52:32.748644 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:32.748604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:52:42.531861 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:42.531821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:52:44.422832 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.422794    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" podUID="4886654d-28d6-4d39-b9fd-45596b6015f5" containerName="main" containerID="cri-o://c857580555f1ea5e848eb15c3fce968d39cc79f0bab7d2dbf7edc322562778a8" gracePeriod=45
Sep 10 19:52:44.576390 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.576359    2567 generic.go:356] "Generic (PLEG): container finished" podID="4886654d-28d6-4d39-b9fd-45596b6015f5" containerID="c857580555f1ea5e848eb15c3fce968d39cc79f0bab7d2dbf7edc322562778a8" exitCode=0
Sep 10 19:52:44.576582 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.576446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" event={"ID":"4886654d-28d6-4d39-b9fd-45596b6015f5","Type":"ContainerDied","Data":"c857580555f1ea5e848eb15c3fce968d39cc79f0bab7d2dbf7edc322562778a8"}
Sep 10 19:52:44.682065 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.681993    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:52:44.758121 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758080    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758175    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758200    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758236    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758344 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758269    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758355    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home" (OuterVolumeSpecName: "home") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:44.758577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsfw9\" (UniqueName: \"kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9\") pod \"4886654d-28d6-4d39-b9fd-45596b6015f5\" (UID: \"4886654d-28d6-4d39-b9fd-45596b6015f5\") "
Sep 10 19:52:44.758577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758511    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:44.758577 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758534    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:44.758745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758673    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-tmp-dir\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:44.758745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758684    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-model-cache\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:44.758745 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.758692    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-home\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:44.760502 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.760480    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 19:52:44.760502 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.760493    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9" (OuterVolumeSpecName: "kube-api-access-wsfw9") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "kube-api-access-wsfw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 19:52:44.760619 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.760572    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm" (OuterVolumeSpecName: "dshm") pod "4886654d-28d6-4d39-b9fd-45596b6015f5" (UID: "4886654d-28d6-4d39-b9fd-45596b6015f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 19:52:44.859545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.859493    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsfw9\" (UniqueName: \"kubernetes.io/projected/4886654d-28d6-4d39-b9fd-45596b6015f5-kube-api-access-wsfw9\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:44.859545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.859534    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4886654d-28d6-4d39-b9fd-45596b6015f5-tls-certs\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:44.859545 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:44.859550    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4886654d-28d6-4d39-b9fd-45596b6015f5-dshm\") on node \"ip-10-0-134-26.ec2.internal\" DevicePath \"\""
Sep 10 19:52:45.583470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.583438    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"
Sep 10 19:52:45.583470 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.583462    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb" event={"ID":"4886654d-28d6-4d39-b9fd-45596b6015f5","Type":"ContainerDied","Data":"dfa136a464cf318d38c48383a63237c28b57493b1ea2a74e63df9b598cdc7665"}
Sep 10 19:52:45.584031 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.583508    2567 scope.go:117] "RemoveContainer" containerID="c857580555f1ea5e848eb15c3fce968d39cc79f0bab7d2dbf7edc322562778a8"
Sep 10 19:52:45.606071 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.606038    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:52:45.611121 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.611087    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-2517a652-kserve-6c7cdc88b7-rlplb"]
Sep 10 19:52:45.702234 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:45.702192    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4886654d-28d6-4d39-b9fd-45596b6015f5" path="/var/lib/kubelet/pods/4886654d-28d6-4d39-b9fd-45596b6015f5/volumes"
Sep 10 19:52:49.019690 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:49.019646    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:52:56.640160 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.640127    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:52:56.764312 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.764270    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"]
Sep 10 19:52:56.764693 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.764680    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4886654d-28d6-4d39-b9fd-45596b6015f5" containerName="main"
Sep 10 19:52:56.764741 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.764695    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4886654d-28d6-4d39-b9fd-45596b6015f5" containerName="main"
Sep 10 19:52:56.764789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.764780    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4886654d-28d6-4d39-b9fd-45596b6015f5" containerName="main"
Sep 10 19:52:56.768057 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.768036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.770403 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.770370    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 19:52:56.770573 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.770370    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 19:52:56.771075 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.771051    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 19:52:56.771306 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.771260    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gr54c\""
Sep 10 19:52:56.771417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.771393    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 19:52:56.778600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.778564    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"]
Sep 10 19:52:56.871554 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.871510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-openshift-service-ca\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.871733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.871609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-maas-api-tls\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.871733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.871656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-metrics-certs\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.871733 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.871692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wksjw\" (UniqueName: \"kubernetes.io/projected/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-kube-api-access-wksjw\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.972645 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.972601    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-openshift-service-ca\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.972860 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.972667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-maas-api-tls\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.972860 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.972693    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-metrics-certs\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.972860 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.972722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wksjw\" (UniqueName: \"kubernetes.io/projected/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-kube-api-access-wksjw\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.973424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.973401    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-openshift-service-ca\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.975256 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.975234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-maas-api-tls\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.975423 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.975361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-metrics-certs\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:56.982034 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:56.982006    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wksjw\" (UniqueName: \"kubernetes.io/projected/49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4-kube-api-access-wksjw\") pod \"maas-api-8cb5f7c95-f7ztk\" (UID: \"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4\") " pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:57.080058 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:57.080016    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:57.356994 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:57.356967    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"]
Sep 10 19:52:57.359446 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:52:57.359412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49fc1d6e_5f27_4090_b8a5_9e9345d6a7f4.slice/crio-fdf9b22b0051a34d00987ab3b1a1e227ea035bf8b245c77799af16775ffc7699 WatchSource:0}: Error finding container fdf9b22b0051a34d00987ab3b1a1e227ea035bf8b245c77799af16775ffc7699: Status 404 returned error can't find the container with id fdf9b22b0051a34d00987ab3b1a1e227ea035bf8b245c77799af16775ffc7699
Sep 10 19:52:57.360989 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:57.360962    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 19:52:57.652017 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:57.651980    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk" event={"ID":"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4","Type":"ContainerStarted","Data":"fdf9b22b0051a34d00987ab3b1a1e227ea035bf8b245c77799af16775ffc7699"}
Sep 10 19:52:58.662092 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:58.662049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk" event={"ID":"49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4","Type":"ContainerStarted","Data":"745c94dfe0791f4ddbddcf3e383417b57f3247f294e63da374b620942f8c9aaa"}
Sep 10 19:52:58.662490 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:58.662145    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:52:58.680759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:52:58.680700    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk" podStartSLOduration=2.3442770250000002 podStartE2EDuration="2.680686741s" totalImagesPullingTime="336.409716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:52:58.680283762 +0000 UTC m=+2237.501339894" watchObservedRunningTime="2026-09-10 19:52:58.680686741 +0000 UTC m=+2237.501742875"
Sep 10 19:53:04.125805 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:04.125722    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:53:04.674214 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:04.674185    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8cb5f7c95-f7ztk"
Sep 10 19:53:12.124183 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:12.124146    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:53:20.119719 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:20.119679    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-ls7r4"]
Sep 10 19:53:23.168770 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:23.168735    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d67c4d47-5928k_f7507dba-9d78-49e1-a58a-2662b355f070/authorino/0.log"
Sep 10 19:53:29.403053 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:29.403021    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-wnn9l_a9400e92-698e-412b-a8cd-e22ca15f53c0/copy-manifests/0.log"
Sep 10 19:53:29.410496 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:29.410473    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-wnn9l_a9400e92-698e-412b-a8cd-e22ca15f53c0/manager/0.log"
Sep 10 19:53:29.768444 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:29.768326    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8cb5f7c95-f7ztk_49fc1d6e-5f27-4090-b8a5-9e9345d6a7f4/maas-api/0.log"
Sep 10 19:53:31.002492 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:31.002446    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d67c4d47-5928k_f7507dba-9d78-49e1-a58a-2662b355f070/authorino/0.log"
Sep 10 19:53:31.362346 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:31.362264    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zgwmt_316edcbe-4bcd-4999-aed5-9b87dc2d82f8/kuadrant-console-plugin/0.log"
Sep 10 19:53:31.606758 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:31.606729    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ls7r4_7e8ec980-8f03-4e98-9c8d-c935cab80af7/limitador/0.log"
Sep 10 19:53:32.334398 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:32.334370    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-hn57b_abbec643-f886-4437-9b3e-5514cccb90f9/kube-auth-proxy/0.log"
Sep 10 19:53:32.931628 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:32.931597    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cff896496-rh6zc_cb45bf43-4568-4efe-8593-2c61a1950857/router/0.log"
Sep 10 19:53:33.864880 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:33.864837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vd7tm_86c35c34-c738-4649-a127-1a4be42a29d6/main/0.log"
Sep 10 19:53:33.974773 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:33.974738    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-6lb24_77258ff2-b9c3-4188-a810-a33588c6acc5/main/0.log"
Sep 10 19:53:40.000581 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.000543    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt8w/must-gather-95fmh"]
Sep 10 19:53:40.004542 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.004521    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.007059 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.007032    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8bt8w\"/\"default-dockercfg-ss8rz\""
Sep 10 19:53:40.007206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.007070    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt8w\"/\"openshift-service-ca.crt\""
Sep 10 19:53:40.007206 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.007105    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8bt8w\"/\"kube-root-ca.crt\""
Sep 10 19:53:40.014921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.014889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/must-gather-95fmh"]
Sep 10 19:53:40.072033 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.071992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm8rn\" (UniqueName: \"kubernetes.io/projected/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-kube-api-access-pm8rn\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.072223 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.072069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-must-gather-output\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.172624 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.172588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-must-gather-output\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.172812 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.172669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm8rn\" (UniqueName: \"kubernetes.io/projected/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-kube-api-access-pm8rn\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.173003 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.172983    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-must-gather-output\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.181229 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.181203    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm8rn\" (UniqueName: \"kubernetes.io/projected/ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c-kube-api-access-pm8rn\") pod \"must-gather-95fmh\" (UID: \"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c\") " pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.315409 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.315318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/must-gather-95fmh"
Sep 10 19:53:40.591123 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.591096    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/must-gather-95fmh"]
Sep 10 19:53:40.593244 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:53:40.593211    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecfabcad_c6dc_464f_91a0_e1f5fbbf2c8c.slice/crio-b517593525a13a1c872a5258d0597e8f791958a43254fc3813a7647faa5d6485 WatchSource:0}: Error finding container b517593525a13a1c872a5258d0597e8f791958a43254fc3813a7647faa5d6485: Status 404 returned error can't find the container with id b517593525a13a1c872a5258d0597e8f791958a43254fc3813a7647faa5d6485
Sep 10 19:53:40.910467 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:40.910430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/must-gather-95fmh" event={"ID":"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c","Type":"ContainerStarted","Data":"b517593525a13a1c872a5258d0597e8f791958a43254fc3813a7647faa5d6485"}
Sep 10 19:53:41.921202 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:41.921163    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/must-gather-95fmh" event={"ID":"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c","Type":"ContainerStarted","Data":"63115fc1491e1793887c1e388e7c9c2d9fcf5300df40345e5b4e16a457a4842f"}
Sep 10 19:53:41.921799 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:41.921766    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/must-gather-95fmh" event={"ID":"ecfabcad-c6dc-464f-91a0-e1f5fbbf2c8c","Type":"ContainerStarted","Data":"a762c92d2b3e0b92d1686f5ec196de0304f2c12cc802e16bce976f005878d213"}
Sep 10 19:53:41.943229 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:41.943153    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt8w/must-gather-95fmh" podStartSLOduration=2.178391206 podStartE2EDuration="2.943128103s" totalImagesPullingTime="764.736897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 19:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:41.938759804 +0000 UTC m=+2280.759816056" watchObservedRunningTime="2026-09-10 19:53:41.943128103 +0000 UTC m=+2280.764184236"
Sep 10 19:53:43.082216 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:43.082182    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n7lpb_9ed5274b-92ef-48e0-9466-a04023ae4f93/global-pull-secret-syncer/0.log"
Sep 10 19:53:43.226158 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:43.226126    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wprrs_95578d05-bf3f-4168-8ce1-3e38c9601bc3/konnectivity-agent/0.log"
Sep 10 19:53:43.360972 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:43.360865    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-26.ec2.internal_c918b9dd0fb780c2b3cbfafa24767b4e/haproxy/0.log"
Sep 10 19:53:48.016017 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:48.015981    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79d67c4d47-5928k_f7507dba-9d78-49e1-a58a-2662b355f070/authorino/0.log"
Sep 10 19:53:48.121359 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:48.121332    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zgwmt_316edcbe-4bcd-4999-aed5-9b87dc2d82f8/kuadrant-console-plugin/0.log"
Sep 10 19:53:48.192623 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:48.192598    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-ls7r4_7e8ec980-8f03-4e98-9c8d-c935cab80af7/limitador/0.log"
Sep 10 19:53:49.895991 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:49.895902    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/alertmanager/0.log"
Sep 10 19:53:49.924526 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:49.924447    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/config-reloader/0.log"
Sep 10 19:53:49.947921 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:49.947877    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/kube-rbac-proxy-web/0.log"
Sep 10 19:53:49.974426 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:49.974401    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/kube-rbac-proxy/0.log"
Sep 10 19:53:49.999601 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:49.999569    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/kube-rbac-proxy-metric/0.log"
Sep 10 19:53:50.027587 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.027554    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/prom-label-proxy/0.log"
Sep 10 19:53:50.053635 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.053607    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f8a7a297-4f9d-40fc-a25d-6e9872d302fc/init-config-reloader/0.log"
Sep 10 19:53:50.102116 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.102083    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-64ktg_5a084ee9-b2eb-4e9a-ae5d-45957770d2c4/cluster-monitoring-operator/0.log"
Sep 10 19:53:50.238472 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.238374    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-lt52t_69cbe945-945b-4a5c-aad3-cc58f899888a/monitoring-plugin/0.log"
Sep 10 19:53:50.358460 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.358432    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htcj2_9fdfcb26-6a93-48ba-b9b4-bebb2cefc675/node-exporter/0.log"
Sep 10 19:53:50.383604 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.383569    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htcj2_9fdfcb26-6a93-48ba-b9b4-bebb2cefc675/kube-rbac-proxy/0.log"
Sep 10 19:53:50.409291 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.409254    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-htcj2_9fdfcb26-6a93-48ba-b9b4-bebb2cefc675/init-textfile/0.log"
Sep 10 19:53:50.535295 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.535196    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6cpxs_a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823/kube-rbac-proxy-main/0.log"
Sep 10 19:53:50.566601 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.566574    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6cpxs_a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823/kube-rbac-proxy-self/0.log"
Sep 10 19:53:50.587604 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.587573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-6cpxs_a8ae28ab-8073-4ab4-ab4c-f5a5b3dfd823/openshift-state-metrics/0.log"
Sep 10 19:53:50.912759 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.912732    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-xt6kj_51e9ada4-7ce2-4c09-afdd-5d3883faee76/prometheus-operator-admission-webhook/0.log"
Sep 10 19:53:50.973968 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:50.970406    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-75d6f8c69b-f9psk_6d1969e6-1486-43b5-84eb-a1364012647b/telemeter-client/0.log"
Sep 10 19:53:51.010961 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.010680    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-75d6f8c69b-f9psk_6d1969e6-1486-43b5-84eb-a1364012647b/reload/0.log"
Sep 10 19:53:51.040095 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.040048    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-75d6f8c69b-f9psk_6d1969e6-1486-43b5-84eb-a1364012647b/kube-rbac-proxy/0.log"
Sep 10 19:53:51.082860 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.082828    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/thanos-query/0.log"
Sep 10 19:53:51.123849 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.123710    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/kube-rbac-proxy-web/0.log"
Sep 10 19:53:51.151360 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.151317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/kube-rbac-proxy/0.log"
Sep 10 19:53:51.178511 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.178479    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/prom-label-proxy/0.log"
Sep 10 19:53:51.205463 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.205436    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/kube-rbac-proxy-rules/0.log"
Sep 10 19:53:51.234171 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:51.234139    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d8b5d594c-xtt2c_bfdf7929-a3ae-46a5-84ff-0e78602fbb7e/kube-rbac-proxy-metrics/0.log"
Sep 10 19:53:52.706276 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:52.706247    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-mr9td_6e6dbc24-2f34-48d9-9233-26f2ab31da70/networking-console-plugin/0.log"
Sep 10 19:53:53.414634 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:53.414604    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/1.log"
Sep 10 19:53:53.420590 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:53.420558    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-6r9qv_9c8d45b7-a264-43de-a9e8-3d8a4fb57669/console-operator/2.log"
Sep 10 19:53:53.971706 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:53.971669    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66b9dd6998-qv6qz_b6af28e3-88b9-4cd8-a704-6ed31e2515ef/console/0.log"
Sep 10 19:53:54.038792 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.038748    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"]
Sep 10 19:53:54.049356 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.049321    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.052253 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.052224    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"]
Sep 10 19:53:54.139464 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.139423    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-podres\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.139789 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.139771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5hmt\" (UniqueName: \"kubernetes.io/projected/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-kube-api-access-k5hmt\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.140067 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.140042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-sys\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.140271 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.140257    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-proc\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.140424 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.140409    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-lib-modules\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241272    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-sys\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-proc\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241366 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-lib-modules\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-podres\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241445    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-proc\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5hmt\" (UniqueName: \"kubernetes.io/projected/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-kube-api-access-k5hmt\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241527    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-podres\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-sys\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.241653 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.241548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-lib-modules\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.250925 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.250893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5hmt\" (UniqueName: \"kubernetes.io/projected/e582d8e5-3d7e-4d86-9909-0abb4b6200c9-kube-api-access-k5hmt\") pod \"perf-node-gather-daemonset-6hbzw\" (UID: \"e582d8e5-3d7e-4d86-9909-0abb4b6200c9\") " pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.364349 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.364297    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:54.666417 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.666378    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"]
Sep 10 19:53:54.675742 ip-10-0-134-26 kubenswrapper[2567]: W0910 19:53:54.675705    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode582d8e5_3d7e_4d86_9909_0abb4b6200c9.slice/crio-3208d3cd29b28bcddaa79b88daccbf3017f10e133dfe25f1e0b46ab9f01c1842 WatchSource:0}: Error finding container 3208d3cd29b28bcddaa79b88daccbf3017f10e133dfe25f1e0b46ab9f01c1842: Status 404 returned error can't find the container with id 3208d3cd29b28bcddaa79b88daccbf3017f10e133dfe25f1e0b46ab9f01c1842
Sep 10 19:53:54.766180 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:54.766154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-rpwzs_ae7517b3-e699-4658-8e2a-919d98953038/volume-data-source-validator/0.log"
Sep 10 19:53:55.041447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.041344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw" event={"ID":"e582d8e5-3d7e-4d86-9909-0abb4b6200c9","Type":"ContainerStarted","Data":"7275d570b168ea492f22af2c7a8a62ef83f6b48a4a8285b845cbc9e77feaa3aa"}
Sep 10 19:53:55.041447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.041420    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:53:55.041447 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.041446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw" event={"ID":"e582d8e5-3d7e-4d86-9909-0abb4b6200c9","Type":"ContainerStarted","Data":"3208d3cd29b28bcddaa79b88daccbf3017f10e133dfe25f1e0b46ab9f01c1842"}
Sep 10 19:53:55.066956 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.066890    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw" podStartSLOduration=1.066873288 podStartE2EDuration="1.066873288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 19:53:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 19:53:55.064129789 +0000 UTC m=+2293.885185923" watchObservedRunningTime="2026-09-10 19:53:55.066873288 +0000 UTC m=+2293.887929420"
Sep 10 19:53:55.952338 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.952302    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-psnrk_e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2/dns/0.log"
Sep 10 19:53:55.976193 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:55.976167    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-psnrk_e7e23b75-a206-4a9f-9fe6-e2d861a3a2d2/kube-rbac-proxy/0.log"
Sep 10 19:53:56.056611 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:56.056582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ck8rf_df856401-4913-4059-9bce-5bdf0655c1bd/dns-node-resolver/0.log"
Sep 10 19:53:56.768428 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:56.768390    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lbmvr_264a5601-4ce3-4a62-9baf-0f9c175a4244/node-ca/0.log"
Sep 10 19:53:58.285430 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:58.285401    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-hn57b_abbec643-f886-4437-9b3e-5514cccb90f9/kube-auth-proxy/0.log"
Sep 10 19:53:58.449657 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:58.449623    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cff896496-rh6zc_cb45bf43-4568-4efe-8593-2c61a1950857/router/0.log"
Sep 10 19:53:59.358038 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:59.358003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-skmtm_e5bf6c7c-9f53-4bf9-ab7f-21e695929503/serve-healthcheck-canary/0.log"
Sep 10 19:53:59.924317 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:59.924284    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8rgkr_a29dd7c6-d869-4497-9a3a-23a7289a6c58/insights-operator/0.log"
Sep 10 19:53:59.924741 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:53:59.924726    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-8rgkr_a29dd7c6-d869-4497-9a3a-23a7289a6c58/insights-operator/1.log"
Sep 10 19:54:00.115085 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:00.115022    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zch66_38bd8fb3-555c-4b1e-9ee7-6663278dd749/kube-rbac-proxy/0.log"
Sep 10 19:54:00.149108 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:00.147486    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zch66_38bd8fb3-555c-4b1e-9ee7-6663278dd749/exporter/0.log"
Sep 10 19:54:00.171451 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:00.171418    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zch66_38bd8fb3-555c-4b1e-9ee7-6663278dd749/extractor/0.log"
Sep 10 19:54:01.062889 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:01.062859    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8bt8w/perf-node-gather-daemonset-6hbzw"
Sep 10 19:54:02.947597 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:02.947546    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-wnn9l_a9400e92-698e-412b-a8cd-e22ca15f53c0/manager/0.log"
Sep 10 19:54:02.975520 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:02.975436    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55556678b8-wnn9l_a9400e92-698e-412b-a8cd-e22ca15f53c0/copy-manifests/0.log"
Sep 10 19:54:04.440701 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:04.440671    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77f5df88d-mcqmw_dcd1933a-2b2f-430d-887f-92f1c3d2a573/manager/0.log"
Sep 10 19:54:05.002715 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:05.002683    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-hn57b_abbec643-f886-4437-9b3e-5514cccb90f9/kube-auth-proxy/0.log"
Sep 10 19:54:05.128318 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:05.128286    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cff896496-rh6zc_cb45bf43-4568-4efe-8593-2c61a1950857/router/0.log"
Sep 10 19:54:07.138384 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:07.138341    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64954c75b6-hn57b_abbec643-f886-4437-9b3e-5514cccb90f9/kube-auth-proxy/0.log"
Sep 10 19:54:07.269113 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:07.269075    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cff896496-rh6zc_cb45bf43-4568-4efe-8593-2c61a1950857/router/0.log"
Sep 10 19:54:10.277133 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:10.277101    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dxjsb_ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b/kube-storage-version-migrator-operator/1.log"
Sep 10 19:54:10.278152 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:10.278127    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dxjsb_ae2d4d25-6916-4e33-81f8-dd4a3f0ba92b/kube-storage-version-migrator-operator/0.log"
Sep 10 19:54:11.604411 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.604376    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/kube-multus-additional-cni-plugins/0.log"
Sep 10 19:54:11.629172 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.629074    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/egress-router-binary-copy/0.log"
Sep 10 19:54:11.652045 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.652010    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/cni-plugins/0.log"
Sep 10 19:54:11.674001 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.673971    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/bond-cni-plugin/0.log"
Sep 10 19:54:11.698549 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.698519    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/routeoverride-cni/0.log"
Sep 10 19:54:11.721605 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.721580    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/whereabouts-cni-bincopy/0.log"
Sep 10 19:54:11.744487 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.744460    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxv8f_5817f7ec-d60b-4f3c-9e24-8aa71b716536/whereabouts-cni/0.log"
Sep 10 19:54:11.851635 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.851596    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fzhnf_24170250-ffbb-4c0d-b409-e47efe4993be/kube-multus/0.log"
Sep 10 19:54:11.923967 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.923915    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6qj78_f535af16-3f8b-43f9-9c39-59b1a86b9080/network-metrics-daemon/0.log"
Sep 10 19:54:11.943600 ip-10-0-134-26 kubenswrapper[2567]: I0910 19:54:11.943568    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6qj78_f535af16-3f8b-43f9-9c39-59b1a86b9080/kube-rbac-proxy/0.log"