Sep 10 22:56:12.948874 ip-10-0-137-120 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 22:56:12.948890 ip-10-0-137-120 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 22:56:12.948899 ip-10-0-137-120 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 22:56:12.949254 ip-10-0-137-120 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 22:56:23.170791 ip-10-0-137-120 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 22:56:23.170812 ip-10-0-137-120 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 21b5b8288f5345a3b9c772c7985129d6 --
Sep 10 22:57:56.380502 ip-10-0-137-120 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 22:57:56.824572 ip-10-0-137-120 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:57:56.824572 ip-10-0-137-120 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 22:57:56.824572 ip-10-0-137-120 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:57:56.824572 ip-10-0-137-120 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 22:57:56.824572 ip-10-0-137-120 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:57:56.825763 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.825632    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 22:57:56.829862 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829842    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.829862 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829862    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.829862 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829866    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829869    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829873    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829876    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829880    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829883    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829886    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829890    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829895    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829898    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829902    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829904    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829907    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829909    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829912    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829914    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829917    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829920    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829922    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829925    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.829961 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829928    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829931    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829933    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829936    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829946    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829949    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829951    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829954    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829956    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829959    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829962    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829965    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829968    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829971    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829974    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829977    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829979    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829982    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829984    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829987    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.830470 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829989    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829992    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829994    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829997    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.829999    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830002    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830004    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830007    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830009    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830012    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830015    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830017    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830020    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830023    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830027    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830031    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830034    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830037    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830041    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.830954 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830044    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830046    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830049    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830051    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830053    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830056    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830058    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830061    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830064    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830066    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830068    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830071    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830074    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830076    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830079    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830081    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830084    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830086    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830089    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830091    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.831442 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830093    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830096    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830098    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830102    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830105    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830108    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830111    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830114    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830117    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830137    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830141    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830144    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830147    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830150    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830152    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830155    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830158    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830161    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.831952 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830854    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830860    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830863    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830866    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830869    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830872    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830875    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830877    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830880    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830882    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830885    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830887    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830890    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830893    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830896    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830899    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830901    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830904    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830907    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830910    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.832440 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830912    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830915    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830918    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830920    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830923    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830926    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830929    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830931    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830934    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830936    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830939    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830941    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830946    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830949    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830952    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830955    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830958    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830961    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830963    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.832941 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830966    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830969    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830971    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830973    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830976    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830978    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830981    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830984    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.830986    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831000    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831004    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831006    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831010    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831013    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831016    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831018    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831021    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831024    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831026    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831029    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.833503 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831032    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831034    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831037    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831040    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831043    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831047    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831050    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831053    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831055    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831057    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831060    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831063    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831065    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831068    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831070    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831073    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831075    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831078    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831080    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831083    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.833990 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831085    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831088    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831090    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831092    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831095    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831098    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831103    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831107    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831110    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831112    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831114    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831117    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831139    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831143    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831146    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831148    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831151    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831153    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831156    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.834496 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831158    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831544    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831550    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831554    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831558    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831561    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831564    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831567    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831569    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831572    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831575    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831577    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831580    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831582    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831585    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831587    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831590    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831593    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831596    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831598    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.834956 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831601    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831604    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831606    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831609    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831611    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831614    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831616    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831618    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831621    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831623    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831626    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831628    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831630    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831633    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831636    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831639    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831641    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831644    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831647    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831649    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.835472 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831652    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831654    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831657    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831661    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831664    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831670    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831672    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831675    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831678    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831680    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831683    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831686    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831688    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831691    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831693    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831696    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831699    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831701    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831704    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831706    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.835968 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831709    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831712    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831714    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831717    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831719    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831722    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831724    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831728    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831731    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831734    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831737    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831739    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831742    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831744    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831747    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831751    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831754    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831758    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831761    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.836475 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831764    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831766    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831769    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831771    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831774    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831777    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831779    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831782    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831784    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831787    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831789    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831792    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831794    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831796    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831799    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831801    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831804    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831806    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831808    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831811    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.836948 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.831817    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831902    2565 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831940    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831947    2565 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831952    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831956    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831959    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831964    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831969    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831973    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831977    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831980    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831984    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831987    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831990    2565 flags.go:64] FLAG: --cgroup-root=""
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831992    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831995    2565 flags.go:64] FLAG: --client-ca-file=""
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.831998    2565 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832002    2565 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832006    2565 flags.go:64] FLAG: --cluster-domain=""
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832009    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832012    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832015    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832018    2565 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 22:57:56.837486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832022    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832024    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832027    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832030    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832033    2565 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832036    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832039    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832042    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832046    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832050    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832053    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832056    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832059    2565 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832062    2565 flags.go:64] FLAG: --enable-server="true"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832065    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832069    2565 flags.go:64] FLAG: --event-burst="100"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832072    2565 flags.go:64] FLAG: --event-qps="50"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832075    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832080    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832082    2565 flags.go:64] FLAG: --eviction-hard=""
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832086    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832089    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832092    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832095    2565 flags.go:64] FLAG: --eviction-soft=""
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832098    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 22:57:56.838054 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832100    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832103    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832106    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832109    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832112    2565 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832115    2565 flags.go:64] FLAG: --feature-gates=""
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832135    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832139    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832142    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832145    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832148    2565 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832151    2565 flags.go:64] FLAG: --help="false"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832154    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-137-120.ec2.internal"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832157    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832160    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832164    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832167    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832170    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832173    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832176    2565 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832178    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832181    2565 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832184    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832187    2565 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 22:57:56.838703 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832190    2565 flags.go:64] FLAG: --kube-reserved=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832194    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832196    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832199    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832202    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832205    2565 flags.go:64] FLAG: --lock-file=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832207    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832211    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832214    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832219    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832222    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832226    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832228    2565 flags.go:64] FLAG: --logging-format="text"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832232    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832235    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832238    2565 flags.go:64] FLAG: --manifest-url=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832240    2565 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832244    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832247    2565 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832251    2565 flags.go:64] FLAG: --max-pods="110"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832254    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832257    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832259    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832263    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832266    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 22:57:56.839363 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832269    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832272    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832278    2565 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832281    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832285    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832287    2565 flags.go:64] FLAG: --pod-cidr=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832290    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832295    2565 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832298    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832303    2565 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832305    2565 flags.go:64] FLAG: --port="10250"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832308    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832311    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e0a9c40cc6659c5"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832314    2565 flags.go:64] FLAG: --qos-reserved=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832317    2565 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832320    2565 flags.go:64] FLAG: --register-node="true"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832323    2565 flags.go:64] FLAG: --register-with-taints=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832327    2565 flags.go:64] FLAG: --registry-burst="10"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832330    2565 flags.go:64] FLAG: --registry-qps="5"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832333    2565 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832336    2565 flags.go:64] FLAG: --reserved-memory=""
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832339    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832342    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832345    2565 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832348    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 22:57:56.839982 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832351    2565 flags.go:64] FLAG: --runonce="false"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832353    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832356    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832359    2565 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832362    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832366    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832369    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832372    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832375    2565 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832378    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832381    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832383    2565 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832386    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832389    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832392    2565 flags.go:64] FLAG: --system-cgroups=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832395    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832401    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832404    2565 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832406    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832411    2565 flags.go:64] FLAG: --tls-min-version=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832414    2565 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832416    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832419    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832422    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832425    2565 flags.go:64] FLAG: --v="2"
Sep 10 22:57:56.840608 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832429    2565 flags.go:64] FLAG: --version="false"
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832438    2565 flags.go:64] FLAG: --vmodule=""
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832443    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832446    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832559    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832563    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832566    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832569    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832572    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832575    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832578    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832581    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832585    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832588    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832590    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832593    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832596    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832598    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832601    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832603    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.841286 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832606    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832609    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832611    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832614    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832617    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832620    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832622    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832625    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832627    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832630    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832632    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832635    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832637    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832640    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832642    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832645    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832648    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832651    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832653    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832656    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.841787 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832658    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832661    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832663    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832666    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832670    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832674    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832678    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832681    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832683    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832686    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832688    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832691    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832694    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832696    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832699    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832702    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832705    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832707    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832710    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832712    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.842317 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832715    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832717    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832720    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832723    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832727    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832730    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832733    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832736    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832740    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832743    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832745    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832748    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832750    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832753    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832756    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832758    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832762    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832764    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832767    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.842824 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832769    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832772    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832775    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832777    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832780    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832783    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832785    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832790    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832795    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832799    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832803    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832806    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832809    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832812    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832815    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832818    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832820    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832823    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832825    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.843326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832828    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832831    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832833    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832836    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.832839    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.832847    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.840254    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.840270    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.841086    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.841093    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841171    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841177    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841180    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841185    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.843796 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841190    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841194    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841197    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841200    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841203    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841206    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841208    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841211    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841214    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841216    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841219    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841222    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841226    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841228    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841231    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841233    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841236    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841239    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841241    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841244    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.844169 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841248    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841251    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841254    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841257    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841260    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841264    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841267    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841270    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841274    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841277    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841279    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841282    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841284    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841287    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841290    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841292    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841295    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841298    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841300    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.844713 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841303    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841306    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841308    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841311    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841313    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841316    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841319    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841322    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841325    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841327    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841329    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841332    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841334    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841338    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841340    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841343    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841346    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841349    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841351    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.845266 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841354    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841357    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841360    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841362    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841365    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841368    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841371    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841373    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841376    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841378    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841380    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841383    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841386    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841388    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841391    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841393    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841396    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841398    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841401    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841403    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.845743 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841406    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841409    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841412    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841414    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841416    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841419    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841421    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841424    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841427    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841429    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841432    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841435    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841439    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841442    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841446    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841449    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.846345 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841451    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.841457    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841581    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841586    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841589    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841592    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841595    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841599    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841603    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841606    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841609    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841612    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841615    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841618    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:57:56.846746 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841620    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841623    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841625    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841628    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841631    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841633    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841636    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841638    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841641    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841644    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841646    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841648    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841651    2565 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841654    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841657    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841659    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841662    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841664    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841667    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841669    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:57:56.847100 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841672    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841676    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841679    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841682    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841685    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841687    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841690    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841692    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841695    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841697    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841700    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841703    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841705    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841708    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841710    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841713    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841716    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841718    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841720    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841723    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:57:56.847615 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841726    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841728    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841731    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841733    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841736    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841738    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841741    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841743    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841746    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841748    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841751    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841753    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841756    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841759    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841763    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841766    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841769    2565 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841772    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841775    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841777    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:57:56.848092 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841781    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841783    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841786    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841789    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841792    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841795    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841797    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841800    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841802    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841805    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841808    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841810    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841815    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841817    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841820    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841822    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841825    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841828    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841830    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841833    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:57:56.848591 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841836    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841838    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841841    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841843    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841845    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841848    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:56.841851    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.841856    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.841970    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.844681    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.845788    2565 server.go:1019] "Starting client certificate rotation"
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.845888    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.845923    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 22:57:56.849109 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.848718    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 22:57:56.855570 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.855545    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 22:57:56.869274 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.869252    2565 log.go:25] "Validated CRI v1 runtime API"
Sep 10 22:57:56.872889 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.872871    2565 log.go:25] "Validated CRI v1 image API"
Sep 10 22:57:56.873754 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.873738    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 22:57:56.877233 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.877214    2565 fs.go:135] Filesystem UUIDs: map[35567858-19f2-476c-ac2c-d0a20cbd95d7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 82c88b99-221a-4328-a4f7-83ecbf5e2359:/dev/nvme0n1p4]
Sep 10 22:57:56.877278 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.877234    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 22:57:56.882823 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.882696    2565 manager.go:216] Machine: {Timestamp:2026-09-10 22:57:56.881260609 +0000 UTC m=+0.370845980 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100938 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d42e6ca7523200e215085247528b3 SystemUUID:ec2d42e6-ca75-2320-0e21-5085247528b3 BootID:21b5b828-8f53-45a3-b9c7-72c7985129d6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:91:83:f1:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:91:83:f1:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:69:a2:a7:a0:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 22:57:56.882823 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.882817    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 22:57:56.882936 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.882905    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 22:57:56.885894 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.885873    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 22:57:56.886459 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.886433    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 22:57:56.886606 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.886463    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 22:57:56.887349 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.887337    2565 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 22:57:56.887383 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.887351    2565 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 22:57:56.887383 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.887359    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 22:57:56.888234 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.888222    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 22:57:56.889399 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.889388    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 22:57:56.894307 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894293    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 22:57:56.894440 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894430    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 22:57:56.894478 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894465    2565 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 22:57:56.894505 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894481    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 22:57:56.894505 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894495    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 22:57:56.894565 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894508    2565 kubelet.go:398] "Adding apiserver pod source"
Sep 10 22:57:56.894565 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894519    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 22:57:56.894963 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894948    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 22:57:56.895017 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.894973    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 22:57:56.897483 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.897461    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 22:57:56.899795 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.899772    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 22:57:56.899896 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.899814    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 22:57:56.901913 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901896    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 22:57:56.902003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901925    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 22:57:56.902003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901938    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 22:57:56.902003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901948    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 22:57:56.902003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901965    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 22:57:56.902003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.901977    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902004    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902014    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902038    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902048    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902061    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 22:57:56.902203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902070    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 22:57:56.902957 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902945    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 22:57:56.903017 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.902960    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 22:57:56.905204 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.905180    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 22:57:56.905293 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.905216    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 22:57:56.905293 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.905216    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 22:57:56.907841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.907811    2565 server.go:1268] "Started kubelet"
Sep 10 22:57:56.908516 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.908097    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 22:57:56.908516 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.908410    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 22:57:56.909471 ip-10-0-137-120 systemd[1]: Started Kubernetes Kubelet.
Sep 10 22:57:56.911852 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.911665    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 22:57:56.913142 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.913106    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 22:57:56.913278 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.913259    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 22:57:56.913504 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.913494    2565 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 22:57:56.919334 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.919313    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 22:57:56.919969 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.919953    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 22:57:56.920093 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.920035    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:56.920669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920650    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 22:57:56.920669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920650    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 22:57:56.920791 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920683    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 22:57:56.920791 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920755    2565 factory.go:55] Registering systemd factory
Sep 10 22:57:56.920791 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920772    2565 factory.go:223] Registration of the systemd container factory successfully
Sep 10 22:57:56.920791 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920788    2565 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 22:57:56.920912 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.920796    2565 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 22:57:56.921023 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921011    2565 factory.go:153] Registering CRI-O factory
Sep 10 22:57:56.921056 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921025    2565 factory.go:223] Registration of the crio container factory successfully
Sep 10 22:57:56.921094 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921084    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 22:57:56.921148 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921107    2565 factory.go:103] Registering Raw factory
Sep 10 22:57:56.921148 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921139    2565 manager.go:1196] Started watching for new ooms in manager
Sep 10 22:57:56.921533 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.921520    2565 manager.go:349] Starting recovery of all containers
Sep 10 22:57:56.927647 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.927607    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 22:57:56.928083 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.928042    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 22:57:56.928736 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.927612    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-120.ec2.internal.18d417c2af6acbde  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-120.ec2.internal,UID:ip-10-0-137-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-120.ec2.internal,},FirstTimestamp:2026-09-10 22:57:56.907035614 +0000 UTC m=+0.396620983,LastTimestamp:2026-09-10 22:57:56.907035614 +0000 UTC m=+0.396620983,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-120.ec2.internal,}"
Sep 10 22:57:56.932796 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.932782    2565 manager.go:354] Recovery completed
Sep 10 22:57:56.934764 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.934746    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t6xrd"
Sep 10 22:57:56.937258 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.937246    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:56.939162 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939142    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:56.939273 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939187    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:56.939273 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939199    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:56.939769 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939757    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 22:57:56.939769 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939768    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 22:57:56.939867 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939775    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:56.939867 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.939784    2565 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 22:57:56.940549 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.940535    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:56.940619 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.940566    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:56.940619 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.940583    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:56.940871 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.940856    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t6xrd"
Sep 10 22:57:56.941223 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.941094    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-120.ec2.internal.18d417c2b15502db  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-120.ec2.internal,UID:ip-10-0-137-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-120.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-120.ec2.internal,},FirstTimestamp:2026-09-10 22:57:56.939162331 +0000 UTC m=+0.428747702,LastTimestamp:2026-09-10 22:57:56.939162331 +0000 UTC m=+0.428747702,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-120.ec2.internal,}"
Sep 10 22:57:56.942096 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.942084    2565 policy_none.go:49] "None policy: Start"
Sep 10 22:57:56.942096 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.942100    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 22:57:56.942096 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.942116    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 22:57:56.944027 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.944015    2565 policy_none.go:47] "Start"
Sep 10 22:57:56.983879 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.983857    2565 manager.go:337] "Starting Device Plugin manager"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.983907    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.983921    2565 server.go:87] "Starting device plugin registration server"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.984322    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.984336    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.984490    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.984571    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.984580    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.985202    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:56.985249    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:56.998748    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.000071    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.000155    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.000185    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.000234    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 22:57:57.007776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.002533    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:57:57.085103 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.084997    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:57.086008 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.085989    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:57.086088 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.086021    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:57.086088 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.086032    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:57.086088 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.086056    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.093375 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.093359    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.093430 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.093382    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-120.ec2.internal\": node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.100831 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.100803    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"]
Sep 10 22:57:57.100929 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.100875    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:57.102399 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.102383    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:57.102502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.102411    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:57.102502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.102421    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:57.104566 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.104553    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:57.104696 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.104676    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.104741 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.104722    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:57.105292 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105273    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:57.105352 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105299    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:57.105352 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105300    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:57.105352 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105340    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:57.105457 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105317    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:57.105457 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.105382    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:57.107441 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.107428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.107491 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.107454    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:57:57.108157 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.108110    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:57:57.108227 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.108168    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:57:57.108227 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.108177    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:57:57.118388 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.118371    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.122588 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.122559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eeb794d498bff6249be66491c7495c4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-120.ec2.internal\" (UID: \"4eeb794d498bff6249be66491c7495c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.122588 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.122586    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.122732 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.122601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.128768 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.128747    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-120.ec2.internal\" not found" node="ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.132991 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.132975    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-120.ec2.internal\" not found" node="ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.218748 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.218712    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.223064 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eeb794d498bff6249be66491c7495c4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-120.ec2.internal\" (UID: \"4eeb794d498bff6249be66491c7495c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.223165 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.223165 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.223233 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.223233 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223195    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/652ee79574422439ad796771346218c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal\" (UID: \"652ee79574422439ad796771346218c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.223290 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.223188    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eeb794d498bff6249be66491c7495c4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-120.ec2.internal\" (UID: \"4eeb794d498bff6249be66491c7495c4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.318885 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.318842    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.419714 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.419623    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.432022 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.431996    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.437183 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.437160    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:57.520436 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.520383    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.620889 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.620839    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.721312 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.721233    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.821830 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.821793    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.845163 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.845116    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 22:57:57.845583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.845288    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 22:57:57.922316 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:57.922287    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:57.946181 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.946148    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 22:52:58 +0000 UTC" deadline="2028-05-15 16:58:50.814770754 +0000 UTC"
Sep 10 22:57:57.946181 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:57.946182    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14706h0m52.868591926s"
Sep 10 22:57:58.022769 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:58.022703    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:59.277517 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:59.277481    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652ee79574422439ad796771346218c4.slice/crio-b7b44c9236b195489c8d40d948f41517864ade9ca08b2af5bc03ea9926fcee29 WatchSource:0}: Error finding container b7b44c9236b195489c8d40d948f41517864ade9ca08b2af5bc03ea9926fcee29: Status 404 returned error can't find the container with id b7b44c9236b195489c8d40d948f41517864ade9ca08b2af5bc03ea9926fcee29
Sep 10 22:57:59.277805 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:57:59.277778    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eeb794d498bff6249be66491c7495c4.slice/crio-19f61ba69748cbe374032a474397908ca222067e2169a85ff668d999ed7e003d WatchSource:0}: Error finding container 19f61ba69748cbe374032a474397908ca222067e2169a85ff668d999ed7e003d: Status 404 returned error can't find the container with id 19f61ba69748cbe374032a474397908ca222067e2169a85ff668d999ed7e003d
Sep 10 22:57:59.283635 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.283613    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 22:57:59.326898 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:59.326857    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:59.427464 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:57:59.427422    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-120.ec2.internal\" not found"
Sep 10 22:57:59.491860 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.491796    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:57:59.523820 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.523788    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:59.539793 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.539763    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 22:57:59.542012 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.541995    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal"
Sep 10 22:57:59.552789 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.552763    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 22:57:59.587298 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.587261    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:57:59.621655 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:57:59.621623    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:58:00.098776 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.098736    2565 apiserver.go:52] "Watching apiserver"
Sep 10 22:58:00.105478 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.105452    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 22:58:00.105893 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.105866    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-ld8ds","kube-system/global-pull-secret-syncer-b44ws","kube-system/konnectivity-agent-fm9pt","openshift-cluster-node-tuning-operator/tuned-p75hs","openshift-dns/node-resolver-5d2lx","openshift-multus/multus-additional-cni-plugins-98vd8","openshift-network-operator/iptables-alerter-xc2b4","openshift-ovn-kubernetes/ovnkube-node-qmb7n","kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf","openshift-image-registry/node-ca-6rvc4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal","openshift-multus/multus-sdjtg","openshift-multus/network-metrics-daemon-tvrpd"]
Sep 10 22:58:00.108344 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.108322    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.111042 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.110521    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.111042 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.110656    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.111042 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.110734    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7w5h7\""
Sep 10 22:58:00.111042 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.110656    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 22:58:00.111042 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.110944    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.112433 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.112414    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 22:58:00.112606 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.112581    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 22:58:00.112822 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.112807    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.112893 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.112861    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.113135 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.113116    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.113444 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.113427    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2fl7x\""
Sep 10 22:58:00.114690 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.114645    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 22:58:00.114825 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.114811    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 22:58:00.114888 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.114869    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c5h8h\""
Sep 10 22:58:00.115122 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.115105    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.117081 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.117065    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.117164 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.117129    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vsqtb\""
Sep 10 22:58:00.117164 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.117153    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.117530 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.117508    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.119443 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.119423    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.119541 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.119425    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.119763 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.119748    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7mrt8\""
Sep 10 22:58:00.120172 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.120154    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.121898 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.121876    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 22:58:00.122219 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.122198    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4lz8c\""
Sep 10 22:58:00.122638 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.122619    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 22:58:00.123049 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.123035    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.123162 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.123134    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:00.123375 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.123352    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 22:58:00.127223 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.127206    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:00.127334 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.127304    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:00.129234 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.129214    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.129346 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.129312    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:00.131679 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.131658    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.134612 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.134586    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.134701 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.134683    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.134809 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.134789    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6r7l\""
Sep 10 22:58:00.134910 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.134902    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 22:58:00.135889 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.135447    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.136254 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d7zs\" (UniqueName: \"kubernetes.io/projected/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-kube-api-access-9d7zs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.136345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-k8s-cni-cncf-io\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136439 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-etc-kubernetes\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136439 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136393    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-sys\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136439 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136413    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-tuned\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136439 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136426    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81390241-df7f-40c8-a3a2-25e9a1cdd7de-tmp-dir\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136442    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-binary-copy\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-multus\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136502    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-kubelet\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-conf-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136571    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysconfig\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136594    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-var-lib-kubelet\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-tmp\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkzwv\" (UniqueName: \"kubernetes.io/projected/92b38185-1303-40ff-b4b2-009b0d4fffcc-kube-api-access-wkzwv\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136702    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-host\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136736    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81390241-df7f-40c8-a3a2-25e9a1cdd7de-hosts-file\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136785    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136864    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhbqm\" (UniqueName: \"kubernetes.io/projected/841cb625-d4d4-4a37-917c-273376ce4128-kube-api-access-vhbqm\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136915    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-hostroot\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136939    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-modprobe-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.136964 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136962    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx8rr\" (UniqueName: \"kubernetes.io/projected/81390241-df7f-40c8-a3a2-25e9a1cdd7de-kube-api-access-lx8rr\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.136986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137011    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76be4873-a8c2-4397-bb97-1d9efbd07219-iptables-alerter-script\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137057    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-socket-dir-parent\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137101    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-run\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-lib-modules\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137156    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67507844-de1a-4abb-b65c-950cae41f443-konnectivity-ca\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137180    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-system-cni-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137203    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-kubernetes\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76be4873-a8c2-4397-bb97-1d9efbd07219-host-slash\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137250    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-netns\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137272    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-bin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137352    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jznmn\" (UniqueName: \"kubernetes.io/projected/1aaca3f5-57cf-4c20-9db5-f562f8516602-kube-api-access-jznmn\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67507844-de1a-4abb-b65c-950cae41f443-agent-certs\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-cnibin\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.137502 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-systemd\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137467    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-conf\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137496    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsp9r\" (UniqueName: \"kubernetes.io/projected/76be4873-a8c2-4397-bb97-1d9efbd07219-kube-api-access-tsp9r\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-system-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-cnibin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-cni-binary-copy\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-daemon-config\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-multus-certs\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137878    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137893    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 22:58:00.138369 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.137956    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.138980 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.138447    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dbfr7\""
Sep 10 22:58:00.138980 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.138547    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.139646 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.139621    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 22:58:00.140257 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.140238    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 22:58:00.140564 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.140546    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 22:58:00.142210 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.142065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 22:58:00.142210 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.142091    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 22:58:00.142210 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.142095    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-s2kkf\""
Sep 10 22:58:00.142210 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.142168    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 22:58:00.163449 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.163417    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 22:58:00.163585 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.163512    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7nx6"
Sep 10 22:58:00.173036 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.173007    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7nx6"
Sep 10 22:58:00.208229 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.208171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal" event={"ID":"652ee79574422439ad796771346218c4","Type":"ContainerStarted","Data":"b7b44c9236b195489c8d40d948f41517864ade9ca08b2af5bc03ea9926fcee29"}
Sep 10 22:58:00.209186 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.209163    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal" event={"ID":"4eeb794d498bff6249be66491c7495c4","Type":"ContainerStarted","Data":"19f61ba69748cbe374032a474397908ca222067e2169a85ff668d999ed7e003d"}
Sep 10 22:58:00.224691 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.224664    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 22:58:00.228496 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.228470    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:58:00.238202 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-etc-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238211    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-node-log\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhbqm\" (UniqueName: \"kubernetes.io/projected/841cb625-d4d4-4a37-917c-273376ce4128-kube-api-access-vhbqm\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238256    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76be4873-a8c2-4397-bb97-1d9efbd07219-iptables-alerter-script\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238311    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-socket-dir-parent\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67507844-de1a-4abb-b65c-950cae41f443-konnectivity-ca\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-system-cni-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.238386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-netd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238408    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-kubernetes\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238410    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-socket-dir-parent\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67507844-de1a-4abb-b65c-950cae41f443-agent-certs\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238457    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-system-cni-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238502    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbhqn\" (UniqueName: \"kubernetes.io/projected/57459eda-01fd-4913-bb44-fad382e2e2fc-kube-api-access-xbhqn\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238538    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-kubernetes\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w59pw\" (UniqueName: \"kubernetes.io/projected/8284fd76-acaf-4918-a0b3-cb7b06791f81-kube-api-access-w59pw\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238578    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-config\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-system-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-cni-binary-copy\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.238811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238796    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-daemon-config\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-bin\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-k8s-cni-cncf-io\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238986    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-system-cni-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238972    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-sys\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238999    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/76be4873-a8c2-4397-bb97-1d9efbd07219-iptables-alerter-script\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239023    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-sys\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239047    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.238944    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67507844-de1a-4abb-b65c-950cae41f443-konnectivity-ca\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239046    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/33865906-360c-474b-bf79-55d4cf655b07-ovn-node-metrics-cert\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239082    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-k8s-cni-cncf-io\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239124    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlh9d\" (UniqueName: \"kubernetes.io/projected/33865906-360c-474b-bf79-55d4cf655b07-kube-api-access-vlh9d\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239146    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-kubelet\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.239481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239194    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-conf-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239228    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-conf-dir\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239231    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-kubelet\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysconfig\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239269    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-var-lib-kubelet\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239320    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysconfig\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239344    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-multus-daemon-config\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239352    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-var-lib-kubelet\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-tmp\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-kubelet\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81390241-df7f-40c8-a3a2-25e9a1cdd7de-hosts-file\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239470    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239491    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-systemd-units\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-run\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-lib-modules\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239532    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81390241-df7f-40c8-a3a2-25e9a1cdd7de-hosts-file\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.240330 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8284fd76-acaf-4918-a0b3-cb7b06791f81-serviceca\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239576    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aaca3f5-57cf-4c20-9db5-f562f8516602-cni-binary-copy\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-run\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.239639    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76be4873-a8c2-4397-bb97-1d9efbd07219-host-slash\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239716    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-lib-modules\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-netns\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239737    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76be4873-a8c2-4397-bb97-1d9efbd07219-host-slash\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239778    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-netns\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.239780    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:00.739741872 +0000 UTC m=+3.026158798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239800    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-bin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239817    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jznmn\" (UniqueName: \"kubernetes.io/projected/1aaca3f5-57cf-4c20-9db5-f562f8516602-kube-api-access-jznmn\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239845    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-cnibin\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239847    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-bin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239901    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/841cb625-d4d4-4a37-917c-273376ce4128-cnibin\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-netns\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.241072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-ovn\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.239981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-systemd\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240003    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240025    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-kubelet-config\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-conf\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240069    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsp9r\" (UniqueName: \"kubernetes.io/projected/76be4873-a8c2-4397-bb97-1d9efbd07219-kube-api-access-tsp9r\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-cnibin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-systemd\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240160    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-cnibin\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240193    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-multus-certs\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d7zs\" (UniqueName: \"kubernetes.io/projected/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-kube-api-access-9d7zs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-log-socket\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-sysctl-conf\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240463    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-script-lib\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-run-multus-certs\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-etc-kubernetes\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242013 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-tuned\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240560    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81390241-df7f-40c8-a3a2-25e9a1cdd7de-tmp-dir\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-binary-copy\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240610    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-etc-kubernetes\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240628    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240651    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-var-lib-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240676    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240752    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-multus\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240778    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkzwv\" (UniqueName: \"kubernetes.io/projected/92b38185-1303-40ff-b4b2-009b0d4fffcc-kube-api-access-wkzwv\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240799    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8284fd76-acaf-4918-a0b3-cb7b06791f81-host\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240822    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-slash\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240888    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240913    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81390241-df7f-40c8-a3a2-25e9a1cdd7de-tmp-dir\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-host\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-dbus\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.242841 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-host-var-lib-cni-multus\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.240969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-device-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241025    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241028    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241063    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/841cb625-d4d4-4a37-917c-273376ce4128-cni-binary-copy\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241074    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-env-overrides\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241078    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-host\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-hostroot\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241125    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-modprobe-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx8rr\" (UniqueName: \"kubernetes.io/projected/81390241-df7f-40c8-a3a2-25e9a1cdd7de-kube-api-access-lx8rr\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241172    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aaca3f5-57cf-4c20-9db5-f562f8516602-hostroot\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241176    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-systemd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.243642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.241238    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-modprobe-d\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.244062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.243937    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-etc-tuned\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.244119 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.244101    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b38185-1303-40ff-b4b2-009b0d4fffcc-tmp\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.244256 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.244237    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67507844-de1a-4abb-b65c-950cae41f443-agent-certs\") pod \"konnectivity-agent-fm9pt\" (UID: \"67507844-de1a-4abb-b65c-950cae41f443\") " pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.250857 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.250835    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhbqm\" (UniqueName: \"kubernetes.io/projected/841cb625-d4d4-4a37-917c-273376ce4128-kube-api-access-vhbqm\") pod \"multus-additional-cni-plugins-98vd8\" (UID: \"841cb625-d4d4-4a37-917c-273376ce4128\") " pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.253997 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.253872    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:00.253997 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.253901    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:00.253997 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.253904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jznmn\" (UniqueName: \"kubernetes.io/projected/1aaca3f5-57cf-4c20-9db5-f562f8516602-kube-api-access-jznmn\") pod \"multus-sdjtg\" (UID: \"1aaca3f5-57cf-4c20-9db5-f562f8516602\") " pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.253997 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.253915    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:00.253997 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.253991    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:00.75397396 +0000 UTC m=+3.040390899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:00.255864 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.255823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d7zs\" (UniqueName: \"kubernetes.io/projected/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-kube-api-access-9d7zs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.256143 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.256125    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx8rr\" (UniqueName: \"kubernetes.io/projected/81390241-df7f-40c8-a3a2-25e9a1cdd7de-kube-api-access-lx8rr\") pod \"node-resolver-5d2lx\" (UID: \"81390241-df7f-40c8-a3a2-25e9a1cdd7de\") " pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.256411 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.256386    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkzwv\" (UniqueName: \"kubernetes.io/projected/92b38185-1303-40ff-b4b2-009b0d4fffcc-kube-api-access-wkzwv\") pod \"tuned-p75hs\" (UID: \"92b38185-1303-40ff-b4b2-009b0d4fffcc\") " pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.256535 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.256521    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsp9r\" (UniqueName: \"kubernetes.io/projected/76be4873-a8c2-4397-bb97-1d9efbd07219-kube-api-access-tsp9r\") pod \"iptables-alerter-xc2b4\" (UID: \"76be4873-a8c2-4397-bb97-1d9efbd07219\") " pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.341481 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.341448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.341642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.341541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.342099 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.341605    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-var-lib-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.342209 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.341516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-var-lib-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.344536 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344509    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.344656 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8284fd76-acaf-4918-a0b3-cb7b06791f81-host\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.344656 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-slash\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.344724 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.344724 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344665    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8284fd76-acaf-4918-a0b3-cb7b06791f81-host\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.344724 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.344693    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:00.344724 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344695    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.344891 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-dbus\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.344891 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344758    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-device-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.344891 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344852    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.344891 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344890    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-slash\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345048 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.344929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345048 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345002    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.345048 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345022    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345048 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.345028    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:00.845005719 +0000 UTC m=+3.131422649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:00.345221 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345077    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-env-overrides\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345221 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345080    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-dbus\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.345221 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-systemd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345221 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-device-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.345400 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-etc-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345400 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345275    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-systemd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345505 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345473    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-node-log\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345873 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-node-log\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345955 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345825    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-etc-openvswitch\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.345955 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-netd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhqn\" (UniqueName: \"kubernetes.io/projected/57459eda-01fd-4913-bb44-fad382e2e2fc-kube-api-access-xbhqn\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-netd\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.345992    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w59pw\" (UniqueName: \"kubernetes.io/projected/8284fd76-acaf-4918-a0b3-cb7b06791f81-kube-api-access-w59pw\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.346062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346026    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-config\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346062 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346052    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-bin\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/33865906-360c-474b-bf79-55d4cf655b07-ovn-node-metrics-cert\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346139    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-env-overrides\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346166    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlh9d\" (UniqueName: \"kubernetes.io/projected/33865906-360c-474b-bf79-55d4cf655b07-kube-api-access-vlh9d\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-kubelet\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-systemd-units\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346255    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8284fd76-acaf-4918-a0b3-cb7b06791f81-serviceca\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346326    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-netns\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-ovn\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346353    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-cni-bin\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346387    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-kubelet\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346397    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-kubelet-config\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-log-socket\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-script-lib\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346280    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346600    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-netns\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-host-run-ovn-kubernetes\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346648    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-log-socket\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346663    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-systemd-units\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346684    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/33865906-360c-474b-bf79-55d4cf655b07-run-ovn\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346689    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57459eda-01fd-4913-bb44-fad382e2e2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.346827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346744    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/70214487-51cd-4cf7-b0de-7425e698a6ec-kubelet-config\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.347353 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346876    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8284fd76-acaf-4918-a0b3-cb7b06791f81-serviceca\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.347353 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.346899    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-config\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.347353 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.347074    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/33865906-360c-474b-bf79-55d4cf655b07-ovnkube-script-lib\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.348935 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.348882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/33865906-360c-474b-bf79-55d4cf655b07-ovn-node-metrics-cert\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.361252 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.361212    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhqn\" (UniqueName: \"kubernetes.io/projected/57459eda-01fd-4913-bb44-fad382e2e2fc-kube-api-access-xbhqn\") pod \"aws-ebs-csi-driver-node-d57nf\" (UID: \"57459eda-01fd-4913-bb44-fad382e2e2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.361414 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.361314    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlh9d\" (UniqueName: \"kubernetes.io/projected/33865906-360c-474b-bf79-55d4cf655b07-kube-api-access-vlh9d\") pod \"ovnkube-node-qmb7n\" (UID: \"33865906-360c-474b-bf79-55d4cf655b07\") " pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.361512 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.361495    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w59pw\" (UniqueName: \"kubernetes.io/projected/8284fd76-acaf-4918-a0b3-cb7b06791f81-kube-api-access-w59pw\") pod \"node-ca-6rvc4\" (UID: \"8284fd76-acaf-4918-a0b3-cb7b06791f81\") " pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.421491 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.421458    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xc2b4"
Sep 10 22:58:00.429048 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.429027    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sdjtg"
Sep 10 22:58:00.439697 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.439668    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:00.445881 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.445702    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p75hs"
Sep 10 22:58:00.452473 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.452444    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5d2lx"
Sep 10 22:58:00.459198 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.459176    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-98vd8"
Sep 10 22:58:00.467072 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.467047    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf"
Sep 10 22:58:00.475826 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.475805    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6rvc4"
Sep 10 22:58:00.481660 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.481640    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:00.748931 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.748838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:00.749097 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.749013    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:00.749097 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.749086    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:01.749066183 +0000 UTC m=+4.035483122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:00.849540 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.849502    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:00.849672 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:00.849554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:00.849672 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849654    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:00.849672 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849659    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:00.849771 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849684    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:00.849771 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849695    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:00.849771 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849715    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:01.849701602 +0000 UTC m=+4.136118555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:00.849771 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:00.849734    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:01.849722652 +0000 UTC m=+4.136139583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:01.177098 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:01.176999    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:53:00 +0000 UTC" deadline="2028-04-24 00:38:59.658274571 +0000 UTC"
Sep 10 22:58:01.177098 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:01.177050    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14185h40m58.481227636s"
Sep 10 22:58:01.274262 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:58:01.274232    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841cb625_d4d4_4a37_917c_273376ce4128.slice/crio-c00118f0789a2fd3b6d3a16bf02e92da451e7eabfdc26ce65b6dcdf9b1b0aad5 WatchSource:0}: Error finding container c00118f0789a2fd3b6d3a16bf02e92da451e7eabfdc26ce65b6dcdf9b1b0aad5: Status 404 returned error can't find the container with id c00118f0789a2fd3b6d3a16bf02e92da451e7eabfdc26ce65b6dcdf9b1b0aad5
Sep 10 22:58:01.283602 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:58:01.283579    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b38185_1303_40ff_b4b2_009b0d4fffcc.slice/crio-252dda184a235114e054d08a3e45cb1b7c53028300e51a6333cacfd656a2fe8b WatchSource:0}: Error finding container 252dda184a235114e054d08a3e45cb1b7c53028300e51a6333cacfd656a2fe8b: Status 404 returned error can't find the container with id 252dda184a235114e054d08a3e45cb1b7c53028300e51a6333cacfd656a2fe8b
Sep 10 22:58:01.284555 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:58:01.284529    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33865906_360c_474b_bf79_55d4cf655b07.slice/crio-0028cea06886b39851a9175029a92da66e6323c1c145582c30dffe0551acb438 WatchSource:0}: Error finding container 0028cea06886b39851a9175029a92da66e6323c1c145582c30dffe0551acb438: Status 404 returned error can't find the container with id 0028cea06886b39851a9175029a92da66e6323c1c145582c30dffe0551acb438
Sep 10 22:58:01.287670 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:58:01.287646    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aaca3f5_57cf_4c20_9db5_f562f8516602.slice/crio-66712ae73c93dad6b36c381babcf30c73787e01988c50d291313e018e17de492 WatchSource:0}: Error finding container 66712ae73c93dad6b36c381babcf30c73787e01988c50d291313e018e17de492: Status 404 returned error can't find the container with id 66712ae73c93dad6b36c381babcf30c73787e01988c50d291313e018e17de492
Sep 10 22:58:01.756265 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:01.756170    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:01.756431 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.756311    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:01.756431 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.756384    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:03.756365924 +0000 UTC m=+6.042782869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:01.856714 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:01.856674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:01.856877 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:01.856747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:01.856877 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856872    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:01.856979 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856896    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:01.856979 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856923    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:01.856979 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856933    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:03.856914884 +0000 UTC m=+6.143331829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:01.856979 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856937    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:01.857194 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:01.856990    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:03.856974552 +0000 UTC m=+6.143391480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:02.177846 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.177730    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:53:00 +0000 UTC" deadline="2028-05-31 06:38:18.265757232 +0000 UTC"
Sep 10 22:58:02.177846 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.177770    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15079h40m16.087992384s"
Sep 10 22:58:02.203984 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.203955    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:02.204132 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:02.204074    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:02.204390 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.204375    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:02.204467 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:02.204448    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:02.204526 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.204487    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:02.204573 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:02.204547    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:02.228226 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.228188    2565 generic.go:356] "Generic (PLEG): container finished" podID="652ee79574422439ad796771346218c4" containerID="553f246bdba55b45e53ac5abeb4cf526970be72ee9a4c10daaad8c73da9796ca" exitCode=0
Sep 10 22:58:02.228470 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.228437    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal" event={"ID":"652ee79574422439ad796771346218c4","Type":"ContainerDied","Data":"553f246bdba55b45e53ac5abeb4cf526970be72ee9a4c10daaad8c73da9796ca"}
Sep 10 22:58:02.239127 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.239093    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5d2lx" event={"ID":"81390241-df7f-40c8-a3a2-25e9a1cdd7de","Type":"ContainerStarted","Data":"56b552de7e9e03015170107dc8a8db59491e97bc16cec27d0d22518601b901cd"}
Sep 10 22:58:02.256198 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.256161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fm9pt" event={"ID":"67507844-de1a-4abb-b65c-950cae41f443","Type":"ContainerStarted","Data":"e7dc2fdb07bac1636253eec0f05433a0afa142d2fb4938a7ebe22a710b03b0a9"}
Sep 10 22:58:02.265952 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.265917    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xc2b4" event={"ID":"76be4873-a8c2-4397-bb97-1d9efbd07219","Type":"ContainerStarted","Data":"3b0c5d351297b89d27498eb708e0acd0edccf2b7a31d5af6480a159662052591"}
Sep 10 22:58:02.274674 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.274635    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sdjtg" event={"ID":"1aaca3f5-57cf-4c20-9db5-f562f8516602","Type":"ContainerStarted","Data":"66712ae73c93dad6b36c381babcf30c73787e01988c50d291313e018e17de492"}
Sep 10 22:58:02.285697 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.285660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"0028cea06886b39851a9175029a92da66e6323c1c145582c30dffe0551acb438"}
Sep 10 22:58:02.293748 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.293712    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p75hs" event={"ID":"92b38185-1303-40ff-b4b2-009b0d4fffcc","Type":"ContainerStarted","Data":"252dda184a235114e054d08a3e45cb1b7c53028300e51a6333cacfd656a2fe8b"}
Sep 10 22:58:02.307251 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.307166    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerStarted","Data":"c00118f0789a2fd3b6d3a16bf02e92da451e7eabfdc26ce65b6dcdf9b1b0aad5"}
Sep 10 22:58:02.322668 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.322233    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal" event={"ID":"4eeb794d498bff6249be66491c7495c4","Type":"ContainerStarted","Data":"03b035f36c547977bae4e89c59a61ad65ef72ee71b609164ca99afd9b026aebe"}
Sep 10 22:58:02.338997 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.338927    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-120.ec2.internal" podStartSLOduration=3.338906576 podStartE2EDuration="3.338906576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:02.33811044 +0000 UTC m=+4.624527400" watchObservedRunningTime="2026-09-10 22:58:02.338906576 +0000 UTC m=+4.625323524"
Sep 10 22:58:02.358683 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.358642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rvc4" event={"ID":"8284fd76-acaf-4918-a0b3-cb7b06791f81","Type":"ContainerStarted","Data":"99c13ff75ba8b14b7ca8c50bd7ad70920b46a420abc3792c1142c0ada351ac9e"}
Sep 10 22:58:02.374315 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:02.374250    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf" event={"ID":"57459eda-01fd-4913-bb44-fad382e2e2fc","Type":"ContainerStarted","Data":"087cf6cf27df4f369c3dd4966ac1dc718259a9b5f1a7bf4ded7c8daa178f48fc"}
Sep 10 22:58:03.392716 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:03.392678    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal" event={"ID":"652ee79574422439ad796771346218c4","Type":"ContainerStarted","Data":"85512c0ef2174edf6c6076a05ced7667e2e38730b62187ae4a7eabfd4abdb7c2"}
Sep 10 22:58:03.410473 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:03.410414    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-120.ec2.internal" podStartSLOduration=4.410393149 podStartE2EDuration="4.410393149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:03.409222517 +0000 UTC m=+5.695639468" watchObservedRunningTime="2026-09-10 22:58:03.410393149 +0000 UTC m=+5.696810102"
Sep 10 22:58:03.771433 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:03.771334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:03.771601 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.771532    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:03.771601 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.771595    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:07.771577809 +0000 UTC m=+10.057994744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:03.872831 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:03.872772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:03.872831 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:03.872820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:03.873087 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.872943    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:03.873087 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.873005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:07.872987749 +0000 UTC m=+10.159404681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:03.873492 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.873460    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:03.873492 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.873490    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:03.873631 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.873504    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:03.873631 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:03.873564    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:07.873546463 +0000 UTC m=+10.159963403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:04.203828 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:04.203744    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:04.203984 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:04.203886    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:04.204313 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:04.204274    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:04.204422 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:04.204400    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:04.204492 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:04.204478    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:04.204584 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:04.204566    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:06.204137 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:06.204103    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:06.204596 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:06.204236    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:06.204669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:06.204637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:06.204776 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:06.204727    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:06.204859 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:06.204781    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:06.204908 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:06.204862    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:07.805417 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:07.805376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:07.805873 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.805518    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:07.805873 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.805578    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:15.805564223 +0000 UTC m=+18.091981149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:07.906385 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:07.906332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:07.906385 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:07.906381    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:07.906676 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.906505    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:07.906676 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.906567    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:15.906549063 +0000 UTC m=+18.192965994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:07.906923 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.906896    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:07.906923 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.906928    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:07.907118 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.906942    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:07.907118 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:07.907003    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:15.906986668 +0000 UTC m=+18.193403601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:08.204669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:08.204579    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:08.204820 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:08.204792    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:08.204892 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:08.204872    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:08.204941 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:08.204932    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:08.205011 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:08.204994    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:08.205062 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:08.204759    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:10.204217 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:10.204182    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:10.204756 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:10.204373    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:10.204756 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:10.204376    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:10.204756 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:10.204447    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:10.204756 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:10.204585    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:10.204756 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:10.204667    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:12.204642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:12.204529    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:12.204642 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:12.204570    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:12.205188 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:12.204667    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:12.205188 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:12.204675    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:12.205188 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:12.204761    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:12.205188 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:12.204861    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:14.204247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:14.204247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:14.204250    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:14.204426    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:14.204545    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:14.204769 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:14.204648    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:15.865219 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:15.865182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:15.865715 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.865347    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:15.865715 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.865414    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:31.865394775 +0000 UTC m=+34.151811699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:15.966324 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:15.966272    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:15.966324 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:15.966330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966475    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966474    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966502    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966511    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966544    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:58:31.96652362 +0000 UTC m=+34.252940544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:15.966565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:15.966564    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:58:31.96655543 +0000 UTC m=+34.252972360 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:16.204100 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:16.204014    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:16.204234 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:16.204016    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:16.204234 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:16.204128    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:16.204234 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:16.204143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:16.204352 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:16.204248    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:16.204352 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:16.204341    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:18.204664 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:18.204556    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:18.204664 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:18.204602    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:18.205189 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:18.204676    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:18.205189 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:18.204755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:18.205189 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:18.204872    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:18.205189 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:18.204949    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:20.204212 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.204129    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:20.205108 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.204141    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:20.205108 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.204274    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:20.205108 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:20.204268    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:20.205108 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:20.204377    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:20.205108 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:20.204450    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:20.437143 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.437101    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5d2lx" event={"ID":"81390241-df7f-40c8-a3a2-25e9a1cdd7de","Type":"ContainerStarted","Data":"17fdbc19371e84c4be46fefed4c4dc9b4ccb9856a1ea7bc7ef9536dfec5dcc09"}
Sep 10 22:58:20.438950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.438920    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fm9pt" event={"ID":"67507844-de1a-4abb-b65c-950cae41f443","Type":"ContainerStarted","Data":"9888027d38587fcdb7882cb6772e7172b192ebc3a8eca5aa802b96b33a58dedc"}
Sep 10 22:58:20.440750 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.440722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sdjtg" event={"ID":"1aaca3f5-57cf-4c20-9db5-f562f8516602","Type":"ContainerStarted","Data":"4617f15aee8fec6fa488f66c11236352c370c4c13bb42d0c6c8664112c3c923f"}
Sep 10 22:58:20.444183 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444157    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"951e369c963cc3cc9507d6c8f045b8555c13e969da436248921fe5f4dee7a42d"}
Sep 10 22:58:20.444326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"1a5bb248c2362b8fe45f20b487d46476b15639839ff27a775da1c3fb58a8cd71"}
Sep 10 22:58:20.444326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"4a8b5198f7d6679601402216f3656a18a782ed5de6c9f2c9e464dc73a3a9d7b5"}
Sep 10 22:58:20.444442 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"444cfb8b56c0dd3c7e453f31e91677d83a8f06ab9fc412d9260259e3f308aa0a"}
Sep 10 22:58:20.444442 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"9d88cf822f950173aeb811a5f8b0c3b14abe2571e121788fe7ea1871c5a4c081"}
Sep 10 22:58:20.444442 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.444418    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"ad3d8cc98e6aec4c4ad39d0480fee41713a05ed15ba32338fe1feff6474fdb99"}
Sep 10 22:58:20.445930 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.445909    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p75hs" event={"ID":"92b38185-1303-40ff-b4b2-009b0d4fffcc","Type":"ContainerStarted","Data":"9351bcefcd936b2f193fc50009b766f0273ec15dab1b500551eac8bd69853e14"}
Sep 10 22:58:20.447704 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.447681    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="9816ee78de89ad0d857d08481f71607e344db0ca35ddac33508bd480b5eb5d88" exitCode=0
Sep 10 22:58:20.447797 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.447776    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"9816ee78de89ad0d857d08481f71607e344db0ca35ddac33508bd480b5eb5d88"}
Sep 10 22:58:20.449465 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.449441    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6rvc4" event={"ID":"8284fd76-acaf-4918-a0b3-cb7b06791f81","Type":"ContainerStarted","Data":"30f01bcaf12d2493ea6924e2855d9a0dc449aa337dc9108cda02433c8f05f6b5"}
Sep 10 22:58:20.451127 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.451104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf" event={"ID":"57459eda-01fd-4913-bb44-fad382e2e2fc","Type":"ContainerStarted","Data":"df70bb01112aedb8bc5cc8ee0f93c5baba89d0cf90cd69442882adc940a58098"}
Sep 10 22:58:20.454136 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.454094    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5d2lx" podStartSLOduration=4.371091065 podStartE2EDuration="22.454081987s" totalImagesPullingTime="18.082990922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:20.45376607 +0000 UTC m=+22.740183017" watchObservedRunningTime="2026-09-10 22:58:20.454081987 +0000 UTC m=+22.740498934"
Sep 10 22:58:20.469944 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.469837    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6rvc4" podStartSLOduration=4.385318051 podStartE2EDuration="22.469819041s" totalImagesPullingTime="18.08450099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:20.469363613 +0000 UTC m=+22.755780561" watchObservedRunningTime="2026-09-10 22:58:20.469819041 +0000 UTC m=+22.756236002"
Sep 10 22:58:20.515509 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.515443    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sdjtg" podStartSLOduration=4.363736794 podStartE2EDuration="22.515424591s" totalImagesPullingTime="18.151687797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:20.515409739 +0000 UTC m=+22.801826718" watchObservedRunningTime="2026-09-10 22:58:20.515424591 +0000 UTC m=+22.801841577"
Sep 10 22:58:20.533193 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.533141    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fm9pt" podStartSLOduration=11.836944171 podStartE2EDuration="22.533123665s" totalImagesPullingTime="10.696179494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:20.532416644 +0000 UTC m=+22.818833591" watchObservedRunningTime="2026-09-10 22:58:20.533123665 +0000 UTC m=+22.819540609"
Sep 10 22:58:20.549117 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:20.549071    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p75hs" podStartSLOduration=4.452668258 podStartE2EDuration="22.549054529s" totalImagesPullingTime="18.096386271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:20.549031523 +0000 UTC m=+22.835448473" watchObservedRunningTime="2026-09-10 22:58:20.549054529 +0000 UTC m=+22.835471527"
Sep 10 22:58:21.131001 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.130854    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 22:58:21.206534 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.206427    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T22:58:21.130994316Z","UUID":"c25dfa2a-11a4-469a-947c-e619c4a590d8","Handler":null,"Name":"","Endpoint":""}
Sep 10 22:58:21.207881 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.207860    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 22:58:21.208010 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.207895    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 22:58:21.455025 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.454918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xc2b4" event={"ID":"76be4873-a8c2-4397-bb97-1d9efbd07219","Type":"ContainerStarted","Data":"dfe43258338fb0b9cbc5a5b1a10815b6366ede99c14932644a62ba147301b22e"}
Sep 10 22:58:21.457906 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.457067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf" event={"ID":"57459eda-01fd-4913-bb44-fad382e2e2fc","Type":"ContainerStarted","Data":"3d17112ba5f429469ba3faa770c99910a77258f2e57175b2453239034cc36df4"}
Sep 10 22:58:21.472468 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:21.472409    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xc2b4" podStartSLOduration=5.384371562 podStartE2EDuration="23.472390314s" totalImagesPullingTime="18.088018752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:21.471976133 +0000 UTC m=+23.758393080" watchObservedRunningTime="2026-09-10 22:58:21.472390314 +0000 UTC m=+23.758807264"
Sep 10 22:58:22.204079 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.204046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:22.204079 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.204077    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:22.204313 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:22.204207    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:22.204313 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.204247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:22.204313 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:22.204242    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:22.204423 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:22.204363    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:22.465084 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.465041    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"6ee0486f700baeaba9f54a159d7ca4a5fdf90cae63d6027cdf113f8d0f734880"}
Sep 10 22:58:22.467522 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.467493    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf" event={"ID":"57459eda-01fd-4913-bb44-fad382e2e2fc","Type":"ContainerStarted","Data":"5eb4d8a36d3247bb1f80200e7559a182ef3654a2e033bda7c368815c1b7cb4cf"}
Sep 10 22:58:22.486433 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:22.486378    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d57nf" podStartSLOduration=3.936822858 podStartE2EDuration="24.486359851s" totalImagesPullingTime="20.549536993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:22.486245277 +0000 UTC m=+24.772662224" watchObservedRunningTime="2026-09-10 22:58:22.486359851 +0000 UTC m=+24.772776798"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.203936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.203936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:24.204074    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.203936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:24.204199    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:24.204451 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:24.204351    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:24.218240 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.218133    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:24.219065 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.218885    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:24.478019 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.477561    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" event={"ID":"33865906-360c-474b-bf79-55d4cf655b07","Type":"ContainerStarted","Data":"065e7cd0de0028492f7f4580ec0ea7183a0790b999ddbc5425cf4d32429445a6"}
Sep 10 22:58:24.478019 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.477850    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:24.478019 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.477922    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:24.478270 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.478233    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fm9pt"
Sep 10 22:58:24.497006 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.496975    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:24.511473 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:24.511413    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n" podStartSLOduration=8.348360112 podStartE2EDuration="26.511390272s" totalImagesPullingTime="18.16303016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:24.508367204 +0000 UTC m=+26.794784151" watchObservedRunningTime="2026-09-10 22:58:24.511390272 +0000 UTC m=+26.797807220"
Sep 10 22:58:25.481168 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:25.481139    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:25.481624 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:25.481242    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:25.503134 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:25.503107    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:58:26.203924 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.203893    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:26.203924 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.203910    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:26.204119 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.203999    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:26.204119 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.204012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:26.204240 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.204113    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:26.204240 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.204217    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:26.418258 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.418221    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b44ws"]
Sep 10 22:58:26.422750 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.422701    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ld8ds"]
Sep 10 22:58:26.423348 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.423325    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvrpd"]
Sep 10 22:58:26.484557 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.484518    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="7e2e61c74c1bc4d70be2fbb7afd24b66c2330e57fc0a14b9bd41889b709d31de" exitCode=0
Sep 10 22:58:26.484557 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.484556    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"7e2e61c74c1bc4d70be2fbb7afd24b66c2330e57fc0a14b9bd41889b709d31de"}
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.484600    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.484697    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.484715    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.484772    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:26.484867    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:26.485326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:26.484943    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:27.491981 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:27.491891    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="96b5284e20a1ed799f5006a229e241a49daddb11e2bce7b1292f7b0c0f1fbe01" exitCode=0
Sep 10 22:58:27.492375 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:27.491985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"96b5284e20a1ed799f5006a229e241a49daddb11e2bce7b1292f7b0c0f1fbe01"}
Sep 10 22:58:28.207757 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:28.207727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:28.207757 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:28.207752    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:28.207914 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:28.207727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:28.207914 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:28.207883    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:28.207985 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:28.207973    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:28.208056 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:28.208043    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:28.497313 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:28.496957    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="f84e31cd09bd1dd2d9fc5383eaf3d2c45ef1cd30978c8690b3a5e88ce77b8417" exitCode=0
Sep 10 22:58:28.497313 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:28.497035    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"f84e31cd09bd1dd2d9fc5383eaf3d2c45ef1cd30978c8690b3a5e88ce77b8417"}
Sep 10 22:58:30.204625 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:30.204588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:30.205074 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:30.204588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:30.205074 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:30.204746    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tvrpd" podUID="45d8b65e-7ea0-4ff9-83ad-6da69ade88a2"
Sep 10 22:58:30.205074 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:30.204846    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b44ws" podUID="70214487-51cd-4cf7-b0de-7425e698a6ec"
Sep 10 22:58:30.205074 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:30.204597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:30.205074 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:30.204958    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ld8ds" podUID="7180bef1-77a7-4501-913d-c11f26021635"
Sep 10 22:58:31.883751 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:31.883670    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:31.884280 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.883823    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:31.884280 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.883886    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:03.88387048 +0000 UTC m=+66.170287406 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:58:31.984082 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:31.984042    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:31.984259 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:31.984091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:31.984259 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984213    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:31.984405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984256    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:58:31.984405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984277    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret podName:70214487-51cd-4cf7-b0de-7425e698a6ec nodeName:}" failed. No retries permitted until 2026-09-10 22:59:03.984259188 +0000 UTC m=+66.270676113 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret") pod "global-pull-secret-syncer-b44ws" (UID: "70214487-51cd-4cf7-b0de-7425e698a6ec") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:58:31.984405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984309    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:58:31.984405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984326    2565 projected.go:196] Error preparing data for projected volume kube-api-access-fxsh4 for pod openshift-network-diagnostics/network-check-target-ld8ds: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:31.984405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:31.984377    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4 podName:7180bef1-77a7-4501-913d-c11f26021635 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:03.984361174 +0000 UTC m=+66.270778115 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fxsh4" (UniqueName: "kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4") pod "network-check-target-ld8ds" (UID: "7180bef1-77a7-4501-913d-c11f26021635") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:58:32.011310 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.011270    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-120.ec2.internal" event="NodeReady"
Sep 10 22:58:32.011471 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.011438    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 22:58:32.075981 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.075695    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 22:58:32.095916 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.095886    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 22:58:32.096079 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.095930    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fsc8h"]
Sep 10 22:58:32.096079 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.096023    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.098491 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.098458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 22:58:32.098641 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.098531    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnj2j\""
Sep 10 22:58:32.098784 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.098763    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 22:58:32.098946 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.098931    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 22:58:32.102963 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.102940    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 22:58:32.107664 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.107638    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jnbw5"]
Sep 10 22:58:32.107902 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.107883    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.112382 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.111053    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 22:58:32.112382 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.111248    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 22:58:32.112382 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.111626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-677w4\""
Sep 10 22:58:32.121875 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.121851    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jnbw5"]
Sep 10 22:58:32.121990 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.121884    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fsc8h"]
Sep 10 22:58:32.122077 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.121981    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.125228 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.125205    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2sqgd\""
Sep 10 22:58:32.125396 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.125237    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 22:58:32.125522 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.125216    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 22:58:32.125604 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.125587    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 22:58:32.185336 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185336 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-config-volume\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.185336 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185328    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtgk\" (UniqueName: \"kubernetes.io/projected/7a07f79f-2c5b-44dd-b7c9-556bfe289520-kube-api-access-4xtgk\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.185583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqpgs\" (UniqueName: \"kubernetes.io/projected/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-kube-api-access-kqpgs\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.185583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185516    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-tmp-dir\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.185583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185556    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185808 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185597    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185808 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185636    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185808 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185699    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a07f79f-2c5b-44dd-b7c9-556bfe289520-cert\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.185808 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185720    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.185808 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hbng\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.186026 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.185821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.204561 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.204521    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:58:32.204749 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.204582    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:58:32.204749 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.204521    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:58:32.207016 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.206996    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 22:58:32.207343 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.207323    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 22:58:32.207828 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.207631    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 22:58:32.207828 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.207705    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hw9xb\""
Sep 10 22:58:32.207828 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.207635    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xmnrw\""
Sep 10 22:58:32.208093 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.207884    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 22:58:32.286329 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.286518 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286330    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.286518 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.286465    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:32.286617 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-config-volume\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.286617 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.286526    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:58:32.78650676 +0000 UTC m=+35.072923688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:32.286617 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtgk\" (UniqueName: \"kubernetes.io/projected/7a07f79f-2c5b-44dd-b7c9-556bfe289520-kube-api-access-4xtgk\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286804    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqpgs\" (UniqueName: \"kubernetes.io/projected/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-kube-api-access-kqpgs\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-tmp-dir\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.286974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.287078    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.287093    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287096    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a07f79f-2c5b-44dd-b7c9-556bfe289520-cert\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287125    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.287138    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:58:32.787121238 +0000 UTC m=+35.073538177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-config-volume\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.287326 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287169    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hbng\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.288106 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287417    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-tmp-dir\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.288106 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287424    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.288106 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.288106 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.287860    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.291551 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.291523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.291680 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.291555    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.291783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.291763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a07f79f-2c5b-44dd-b7c9-556bfe289520-cert\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.296924 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.296902    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtgk\" (UniqueName: \"kubernetes.io/projected/7a07f79f-2c5b-44dd-b7c9-556bfe289520-kube-api-access-4xtgk\") pod \"ingress-canary-jnbw5\" (UID: \"7a07f79f-2c5b-44dd-b7c9-556bfe289520\") " pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.297156 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.297131    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.297364 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.297345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqpgs\" (UniqueName: \"kubernetes.io/projected/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-kube-api-access-kqpgs\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.297678 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.297659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hbng\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.432411 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.432379    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jnbw5"
Sep 10 22:58:32.790997 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.790897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:32.791163 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:32.791021    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:32.791163 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.791074    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:32.791163 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.791133    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:32.791163 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.791152    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:32.791163 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.791156    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:58:33.791136825 +0000 UTC m=+36.077553772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:32.791406 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:32.791208    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:58:33.791195454 +0000 UTC m=+36.077612379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:33.768041 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:33.767828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jnbw5"]
Sep 10 22:58:33.773235 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:58:33.773204    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a07f79f_2c5b_44dd_b7c9_556bfe289520.slice/crio-18298d12712b0d3f85e0dbae4947ee238fae8444ece59a590d3a21e0d2ee8929 WatchSource:0}: Error finding container 18298d12712b0d3f85e0dbae4947ee238fae8444ece59a590d3a21e0d2ee8929: Status 404 returned error can't find the container with id 18298d12712b0d3f85e0dbae4947ee238fae8444ece59a590d3a21e0d2ee8929
Sep 10 22:58:33.799692 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:33.799657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:33.799848 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:33.799772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:33.799848 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:33.799812    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:33.799955 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:33.799870    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:33.799955 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:33.799887    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:33.799955 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:33.799894    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:58:35.799871039 +0000 UTC m=+38.086287987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:33.799955 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:33.799929    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:58:35.799917355 +0000 UTC m=+38.086334280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:34.516145 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:34.516106    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="25370ba1251b19327432e8cabf007eb84770e1d6b4ec59fa5aa5c82e610ef023" exitCode=0
Sep 10 22:58:34.516356 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:34.516194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"25370ba1251b19327432e8cabf007eb84770e1d6b4ec59fa5aa5c82e610ef023"}
Sep 10 22:58:34.517716 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:34.517692    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jnbw5" event={"ID":"7a07f79f-2c5b-44dd-b7c9-556bfe289520","Type":"ContainerStarted","Data":"18298d12712b0d3f85e0dbae4947ee238fae8444ece59a590d3a21e0d2ee8929"}
Sep 10 22:58:35.524518 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:35.524477    2565 generic.go:356] "Generic (PLEG): container finished" podID="841cb625-d4d4-4a37-917c-273376ce4128" containerID="9e5e13dd68af92228d2f144048e25fa5cb6737405315332b57efb6eebfc022ad" exitCode=0
Sep 10 22:58:35.525108 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:35.524549    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerDied","Data":"9e5e13dd68af92228d2f144048e25fa5cb6737405315332b57efb6eebfc022ad"}
Sep 10 22:58:35.815162 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:35.815074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:35.815162 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:35.815133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:35.815405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:35.815231    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:35.815405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:35.815236    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:35.815405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:35.815254    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:35.815405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:35.815325    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:58:39.815305841 +0000 UTC m=+42.101722780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:35.815405 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:35.815381    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:58:39.815339412 +0000 UTC m=+42.101756351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:36.529306 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:36.529182    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jnbw5" event={"ID":"7a07f79f-2c5b-44dd-b7c9-556bfe289520","Type":"ContainerStarted","Data":"91da9c7ec517f85f4a10eb248852f0bd12a4c783a3d7592d68ede607bac6cbc6"}
Sep 10 22:58:36.533026 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:36.532989    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98vd8" event={"ID":"841cb625-d4d4-4a37-917c-273376ce4128","Type":"ContainerStarted","Data":"6a4959b21c6ce00acd9cc859a7f811300f7301e0f07cd237f95dbe1cabcf0d2a"}
Sep 10 22:58:36.548325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:36.548255    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jnbw5" podStartSLOduration=2.000952587 podStartE2EDuration="4.548223827s" totalImagesPullingTime="2.54727124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:36.547750036 +0000 UTC m=+38.834166984" watchObservedRunningTime="2026-09-10 22:58:36.548223827 +0000 UTC m=+38.834640774"
Sep 10 22:58:36.573921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:36.573166    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-98vd8" podStartSLOduration=8.197956949 podStartE2EDuration="38.573144201s" totalImagesPullingTime="30.375187252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:58:36.573004028 +0000 UTC m=+38.859420975" watchObservedRunningTime="2026-09-10 22:58:36.573144201 +0000 UTC m=+38.859561149"
Sep 10 22:58:39.847921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:39.847875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:39.847921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:39.847931    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:39.848441 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:39.848031    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:39.848441 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:39.848048    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:39.848441 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:39.848073    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:39.848441 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:39.848085    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:58:47.848069176 +0000 UTC m=+50.134486100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:39.848441 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:39.848135    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:58:47.848116501 +0000 UTC m=+50.134533440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:47.902567 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:47.902523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:47.902593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:47.902684    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:47.902697    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:47.902743    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:47.902762    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:59:03.902747142 +0000 UTC m=+66.189164080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:58:47.903004 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:58:47.902838    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:59:03.90282299 +0000 UTC m=+66.189239917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:58:57.503153 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:58:57.503124    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qmb7n"
Sep 10 22:59:03.920076 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:03.920026    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:03.920085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:03.920110    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.920191    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.920207    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bd556656-6vztw: secret "image-registry-tls" not found
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.920232    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.920273    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls podName:111cbf4e-9628-44f6-b617-a46a60b66c0c nodeName:}" failed. No retries permitted until 2026-09-10 22:59:35.920255294 +0000 UTC m=+98.206672234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls") pod "image-registry-6bd556656-6vztw" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c") : secret "image-registry-tls" not found
Sep 10 22:59:03.920565 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.920310    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls podName:5c0bad51-63d6-45a8-9ce2-dab24b009d0d nodeName:}" failed. No retries permitted until 2026-09-10 22:59:35.920281032 +0000 UTC m=+98.206697957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls") pod "dns-default-fsc8h" (UID: "5c0bad51-63d6-45a8-9ce2-dab24b009d0d") : secret "dns-default-metrics-tls" not found
Sep 10 22:59:03.922839 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:03.922821    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 22:59:03.930227 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.930207    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 22:59:03.930286 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:03.930266    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs podName:45d8b65e-7ea0-4ff9-83ad-6da69ade88a2 nodeName:}" failed. No retries permitted until 2026-09-10 23:00:07.930251308 +0000 UTC m=+130.216668233 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs") pod "network-metrics-daemon-tvrpd" (UID: "45d8b65e-7ea0-4ff9-83ad-6da69ade88a2") : secret "metrics-daemon-secret" not found
Sep 10 22:59:04.020944 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.020908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:59:04.020944 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.020947    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:59:04.024678 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.024660    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 22:59:04.024734 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.024665    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 22:59:04.035209 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.035186    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 22:59:04.035367 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.035348    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/70214487-51cd-4cf7-b0de-7425e698a6ec-original-pull-secret\") pod \"global-pull-secret-syncer-b44ws\" (UID: \"70214487-51cd-4cf7-b0de-7425e698a6ec\") " pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:59:04.044665 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.044643    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxsh4\" (UniqueName: \"kubernetes.io/projected/7180bef1-77a7-4501-913d-c11f26021635-kube-api-access-fxsh4\") pod \"network-check-target-ld8ds\" (UID: \"7180bef1-77a7-4501-913d-c11f26021635\") " pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:59:04.324553 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.324516    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b44ws"
Sep 10 22:59:04.332257 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.332224    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xmnrw\""
Sep 10 22:59:04.340333 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.340310    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:59:04.630325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.630239    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b44ws"]
Sep 10 22:59:04.633901 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:04.633875    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70214487_51cd_4cf7_b0de_7425e698a6ec.slice/crio-b4c804dff8ff9fb76386f259f494620c046c54cde6c6c56d80987c3d682036e3 WatchSource:0}: Error finding container b4c804dff8ff9fb76386f259f494620c046c54cde6c6c56d80987c3d682036e3: Status 404 returned error can't find the container with id b4c804dff8ff9fb76386f259f494620c046c54cde6c6c56d80987c3d682036e3
Sep 10 22:59:04.659891 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:04.659862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ld8ds"]
Sep 10 22:59:04.663236 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:04.663211    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7180bef1_77a7_4501_913d_c11f26021635.slice/crio-67d6b5c3a2a13005c51e1b07d086dc8c768604ce11ca26ba51e4f424f8d7a362 WatchSource:0}: Error finding container 67d6b5c3a2a13005c51e1b07d086dc8c768604ce11ca26ba51e4f424f8d7a362: Status 404 returned error can't find the container with id 67d6b5c3a2a13005c51e1b07d086dc8c768604ce11ca26ba51e4f424f8d7a362
Sep 10 22:59:05.602748 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:05.601940    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b44ws" event={"ID":"70214487-51cd-4cf7-b0de-7425e698a6ec","Type":"ContainerStarted","Data":"b4c804dff8ff9fb76386f259f494620c046c54cde6c6c56d80987c3d682036e3"}
Sep 10 22:59:05.604279 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:05.604239    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ld8ds" event={"ID":"7180bef1-77a7-4501-913d-c11f26021635","Type":"ContainerStarted","Data":"67d6b5c3a2a13005c51e1b07d086dc8c768604ce11ca26ba51e4f424f8d7a362"}
Sep 10 22:59:09.616478 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:09.616435    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ld8ds" event={"ID":"7180bef1-77a7-4501-913d-c11f26021635","Type":"ContainerStarted","Data":"8d519d13787b6ed514b5c5ff1dcf1af55daac47ac58cb2810d0c2f70e04d6358"}
Sep 10 22:59:09.616971 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:09.616526    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:59:09.618053 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:09.618030    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b44ws" event={"ID":"70214487-51cd-4cf7-b0de-7425e698a6ec","Type":"ContainerStarted","Data":"8550ba84d50a2b872a428ca02f0dbc9d268923ff0104ceb5f3752d6191687d97"}
Sep 10 22:59:09.633321 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:09.633261    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ld8ds" podStartSLOduration=66.847272107 podStartE2EDuration="1m11.633247844s" totalImagesPullingTime="4.785975737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:09.632915565 +0000 UTC m=+71.919332536" watchObservedRunningTime="2026-09-10 22:59:09.633247844 +0000 UTC m=+71.919664791"
Sep 10 22:59:09.649085 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:09.649017    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b44ws" podStartSLOduration=66.829981312 podStartE2EDuration="1m11.648999317s" totalImagesPullingTime="4.819018005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:09.64805393 +0000 UTC m=+71.934470877" watchObservedRunningTime="2026-09-10 22:59:09.648999317 +0000 UTC m=+71.935416259"
Sep 10 22:59:13.411012 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.410972    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"]
Sep 10 22:59:13.413882 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.413864    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.416093 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416057    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"]
Sep 10 22:59:13.416420 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416397    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 22:59:13.416541 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-m6hgb\""
Sep 10 22:59:13.416695 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416674    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:59:13.416811 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416754    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 22:59:13.416877 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.416862    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 22:59:13.418196 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.418166    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"]
Sep 10 22:59:13.418319 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.418301    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.420647 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.420626    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"
Sep 10 22:59:13.421834 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.421807    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 22:59:13.421926 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.421846    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 22:59:13.421926 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.421859    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wcxr6\""
Sep 10 22:59:13.421926 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.421881    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 22:59:13.422632 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.422544    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-g8ml8\""
Sep 10 22:59:13.422805 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.422789    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 22:59:13.428745 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.428720    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"]
Sep 10 22:59:13.432231 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.432211    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"]
Sep 10 22:59:13.435998 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.435978    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"]
Sep 10 22:59:13.513265 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.513227    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"]
Sep 10 22:59:13.515908 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.515886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"
Sep 10 22:59:13.518183 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.518152    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:59:13.518347 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.518152    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 22:59:13.518347 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.518274    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bdfkn\""
Sep 10 22:59:13.526476 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.526451    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"]
Sep 10 22:59:13.586040 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586005    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.586040 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q42cz\" (UniqueName: \"kubernetes.io/projected/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-kube-api-access-q42cz\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.586325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11728049-5eea-493c-b523-c9b21283b07e-config\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.586325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586162    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11728049-5eea-493c-b523-c9b21283b07e-serving-cert\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.586325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586239    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55sq4\" (UniqueName: \"kubernetes.io/projected/11728049-5eea-493c-b523-c9b21283b07e-kube-api-access-55sq4\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.586325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586285    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cthcl\" (UniqueName: \"kubernetes.io/projected/0b80daac-304d-4c6c-8d3c-a9cfbc07e19b-kube-api-access-cthcl\") pod \"network-check-source-6f8b768455-bkdqz\" (UID: \"0b80daac-304d-4c6c-8d3c-a9cfbc07e19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"
Sep 10 22:59:13.586472 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.586334    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.687398 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687365    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687418    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q42cz\" (UniqueName: \"kubernetes.io/projected/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-kube-api-access-q42cz\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687438    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11728049-5eea-493c-b523-c9b21283b07e-config\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687471    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11728049-5eea-493c-b523-c9b21283b07e-serving-cert\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687498    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55sq4\" (UniqueName: \"kubernetes.io/projected/11728049-5eea-493c-b523-c9b21283b07e-kube-api-access-55sq4\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:13.687552    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:13.687582 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687578    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cthcl\" (UniqueName: \"kubernetes.io/projected/0b80daac-304d-4c6c-8d3c-a9cfbc07e19b-kube-api-access-cthcl\") pod \"network-check-source-6f8b768455-bkdqz\" (UID: \"0b80daac-304d-4c6c-8d3c-a9cfbc07e19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"
Sep 10 22:59:13.687869 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:13.687639    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:14.187615997 +0000 UTC m=+76.474032939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:13.687869 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687708    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn22k\" (UniqueName: \"kubernetes.io/projected/f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d-kube-api-access-sn22k\") pod \"volume-data-source-validator-86695b7cc7-w82t4\" (UID: \"f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"
Sep 10 22:59:13.687869 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.687746    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.688046 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.688026    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11728049-5eea-493c-b523-c9b21283b07e-config\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.688391 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.688372    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.689888 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.689873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11728049-5eea-493c-b523-c9b21283b07e-serving-cert\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.697837 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.697807    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cthcl\" (UniqueName: \"kubernetes.io/projected/0b80daac-304d-4c6c-8d3c-a9cfbc07e19b-kube-api-access-cthcl\") pod \"network-check-source-6f8b768455-bkdqz\" (UID: \"0b80daac-304d-4c6c-8d3c-a9cfbc07e19b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"
Sep 10 22:59:13.697994 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.697974    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q42cz\" (UniqueName: \"kubernetes.io/projected/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-kube-api-access-q42cz\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:13.698034 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.698016    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55sq4\" (UniqueName: \"kubernetes.io/projected/11728049-5eea-493c-b523-c9b21283b07e-kube-api-access-55sq4\") pod \"service-ca-operator-cf7c776b7-dvflp\" (UID: \"11728049-5eea-493c-b523-c9b21283b07e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.725132 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.725098    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"
Sep 10 22:59:13.736466 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.736441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"
Sep 10 22:59:13.788986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.788944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn22k\" (UniqueName: \"kubernetes.io/projected/f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d-kube-api-access-sn22k\") pod \"volume-data-source-validator-86695b7cc7-w82t4\" (UID: \"f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"
Sep 10 22:59:13.799100 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.799071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn22k\" (UniqueName: \"kubernetes.io/projected/f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d-kube-api-access-sn22k\") pod \"volume-data-source-validator-86695b7cc7-w82t4\" (UID: \"f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"
Sep 10 22:59:13.824910 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.824881    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"
Sep 10 22:59:13.997507 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:13.997426    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp"]
Sep 10 22:59:14.001022 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:14.001000    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11728049_5eea_493c_b523_c9b21283b07e.slice/crio-45b89e251f1867a8fae2e1c31e78b1d3e3236e90a3d1be724bed770419a60da5 WatchSource:0}: Error finding container 45b89e251f1867a8fae2e1c31e78b1d3e3236e90a3d1be724bed770419a60da5: Status 404 returned error can't find the container with id 45b89e251f1867a8fae2e1c31e78b1d3e3236e90a3d1be724bed770419a60da5
Sep 10 22:59:14.020639 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.020614    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz"]
Sep 10 22:59:14.023326 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:14.023283    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b80daac_304d_4c6c_8d3c_a9cfbc07e19b.slice/crio-6a39949678644664a12646db0b008a737ea5e0c7fb56de13a8ca61123418cc6a WatchSource:0}: Error finding container 6a39949678644664a12646db0b008a737ea5e0c7fb56de13a8ca61123418cc6a: Status 404 returned error can't find the container with id 6a39949678644664a12646db0b008a737ea5e0c7fb56de13a8ca61123418cc6a
Sep 10 22:59:14.097179 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.097144    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4"]
Sep 10 22:59:14.101518 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:14.101488    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1dc2ed7_1a8a_4b79_a3e7_3567b1ceb58d.slice/crio-f721b77871e72e36955e544fa8fb4517bb85489bcf22b37371124f323775ab25 WatchSource:0}: Error finding container f721b77871e72e36955e544fa8fb4517bb85489bcf22b37371124f323775ab25: Status 404 returned error can't find the container with id f721b77871e72e36955e544fa8fb4517bb85489bcf22b37371124f323775ab25
Sep 10 22:59:14.191774 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.191737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:14.191976 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:14.191887    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:14.191976 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:14.191953    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:15.19193693 +0000 UTC m=+77.478353859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:14.632830 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.632755    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4" event={"ID":"f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d","Type":"ContainerStarted","Data":"f721b77871e72e36955e544fa8fb4517bb85489bcf22b37371124f323775ab25"}
Sep 10 22:59:14.635016 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.634985    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz" event={"ID":"0b80daac-304d-4c6c-8d3c-a9cfbc07e19b","Type":"ContainerStarted","Data":"ea1e3eb1308bc3d718b471fb5ee2beb19403f1f5936a83223ef22c54cbb28d3f"}
Sep 10 22:59:14.635151 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.635040    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz" event={"ID":"0b80daac-304d-4c6c-8d3c-a9cfbc07e19b","Type":"ContainerStarted","Data":"6a39949678644664a12646db0b008a737ea5e0c7fb56de13a8ca61123418cc6a"}
Sep 10 22:59:14.636388 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.636358    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp" event={"ID":"11728049-5eea-493c-b523-c9b21283b07e","Type":"ContainerStarted","Data":"45b89e251f1867a8fae2e1c31e78b1d3e3236e90a3d1be724bed770419a60da5"}
Sep 10 22:59:14.658899 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:14.658811    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bkdqz" podStartSLOduration=1.658787801 podStartE2EDuration="1.658787801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:14.657515256 +0000 UTC m=+76.943932203" watchObservedRunningTime="2026-09-10 22:59:14.658787801 +0000 UTC m=+76.945204752"
Sep 10 22:59:15.201232 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:15.201155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:15.201424 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:15.201282    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:15.201424 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:15.201368    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:17.201352322 +0000 UTC m=+79.487769247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:15.641867 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:15.641831    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4" event={"ID":"f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d","Type":"ContainerStarted","Data":"74e65eab65beac605baf53b59bbe765af09aeb36f0a4dcf7c08cee3dc8a80aa0"}
Sep 10 22:59:15.658227 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:15.658158    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-w82t4" podStartSLOduration=1.6041761879999998 podStartE2EDuration="2.658136897s" totalImagesPullingTime="1.053960709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:15.656947572 +0000 UTC m=+77.943364520" watchObservedRunningTime="2026-09-10 22:59:15.658136897 +0000 UTC m=+77.944553845"
Sep 10 22:59:16.646870 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:16.646829    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp" event={"ID":"11728049-5eea-493c-b523-c9b21283b07e","Type":"ContainerStarted","Data":"361a698ce215b030aa315a7732dbbd687ea4e6949802fa24a54679ead15a171c"}
Sep 10 22:59:16.666437 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:16.666383    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp" podStartSLOduration=1.770282672 podStartE2EDuration="3.666364286s" totalImagesPullingTime="1.896081614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:16.665895184 +0000 UTC m=+78.952312144" watchObservedRunningTime="2026-09-10 22:59:16.666364286 +0000 UTC m=+78.952781237"
Sep 10 22:59:17.217160 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:17.217105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:17.217375 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:17.217224    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:17.217375 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:17.217278    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:21.217263803 +0000 UTC m=+83.503680728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:18.548947 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.548908    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"]
Sep 10 22:59:18.552030 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.552008    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"
Sep 10 22:59:18.554224 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.554185    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 22:59:18.554938 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.554921    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jwm4t\""
Sep 10 22:59:18.555012 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.554946    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 22:59:18.563110 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.563083    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"]
Sep 10 22:59:18.728396 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.728354    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfxwl\" (UniqueName: \"kubernetes.io/projected/1965f0ec-a915-45e0-b3c4-830431034b52-kube-api-access-sfxwl\") pod \"migrator-8b4d697c5-wz6k9\" (UID: \"1965f0ec-a915-45e0-b3c4-830431034b52\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"
Sep 10 22:59:18.829345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.829161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfxwl\" (UniqueName: \"kubernetes.io/projected/1965f0ec-a915-45e0-b3c4-830431034b52-kube-api-access-sfxwl\") pod \"migrator-8b4d697c5-wz6k9\" (UID: \"1965f0ec-a915-45e0-b3c4-830431034b52\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"
Sep 10 22:59:18.840526 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.840498    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfxwl\" (UniqueName: \"kubernetes.io/projected/1965f0ec-a915-45e0-b3c4-830431034b52-kube-api-access-sfxwl\") pod \"migrator-8b4d697c5-wz6k9\" (UID: \"1965f0ec-a915-45e0-b3c4-830431034b52\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"
Sep 10 22:59:18.864406 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:18.864367    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"
Sep 10 22:59:19.136158 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:19.136089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9"]
Sep 10 22:59:19.139419 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:19.139391    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1965f0ec_a915_45e0_b3c4_830431034b52.slice/crio-0588701f80bc652186ea28356a5f0a38a1cae25a9c845893aacbdf687d90c0e1 WatchSource:0}: Error finding container 0588701f80bc652186ea28356a5f0a38a1cae25a9c845893aacbdf687d90c0e1: Status 404 returned error can't find the container with id 0588701f80bc652186ea28356a5f0a38a1cae25a9c845893aacbdf687d90c0e1
Sep 10 22:59:19.657402 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:19.657360    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9" event={"ID":"1965f0ec-a915-45e0-b3c4-830431034b52","Type":"ContainerStarted","Data":"0588701f80bc652186ea28356a5f0a38a1cae25a9c845893aacbdf687d90c0e1"}
Sep 10 22:59:20.230064 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.230029    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8n5mf"]
Sep 10 22:59:20.234048 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.234017    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.236416 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.236389    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 22:59:20.236566 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.236398    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 22:59:20.237197 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.237172    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 22:59:20.237411 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.237392    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9dvmk\""
Sep 10 22:59:20.237618 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.237602    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 22:59:20.240953 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.240930    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4edd12a4-1332-449e-9c82-b9f8b63002a2-data-volume\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.240953 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.240959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97cvg\" (UniqueName: \"kubernetes.io/projected/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-api-access-97cvg\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.241103 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.241039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4edd12a4-1332-449e-9c82-b9f8b63002a2-crio-socket\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.241144 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.241104    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.241144 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.241133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.245775 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.245744    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8n5mf"]
Sep 10 22:59:20.342485 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4edd12a4-1332-449e-9c82-b9f8b63002a2-crio-socket\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.342610 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.342610 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.342610 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4edd12a4-1332-449e-9c82-b9f8b63002a2-crio-socket\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.342610 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.342597    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:59:20.342852 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.342613    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:20.342852 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.342654    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:20.842640748 +0000 UTC m=+83.129057675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:20.342852 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4edd12a4-1332-449e-9c82-b9f8b63002a2-data-volume\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.342852 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.342691    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:20.842675929 +0000 UTC m=+83.129092876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : configmap "kube-rbac-proxy" not found
Sep 10 22:59:20.342852 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342707    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97cvg\" (UniqueName: \"kubernetes.io/projected/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-api-access-97cvg\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.343030 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.342898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4edd12a4-1332-449e-9c82-b9f8b63002a2-data-volume\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.355133 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.355109    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97cvg\" (UniqueName: \"kubernetes.io/projected/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-api-access-97cvg\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.662516 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.662435    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9" event={"ID":"1965f0ec-a915-45e0-b3c4-830431034b52","Type":"ContainerStarted","Data":"da3a48fde0e4f27993a558e631ab5dae0d6b29d7fa64c67363eb1dd78ea085e1"}
Sep 10 22:59:20.662516 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.662491    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9" event={"ID":"1965f0ec-a915-45e0-b3c4-830431034b52","Type":"ContainerStarted","Data":"794df03628afb059b82a5260c6cb457725e6fbc613a644f07466513c5d99009b"}
Sep 10 22:59:20.680527 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.680477    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-wz6k9" podStartSLOduration=1.521786085 podStartE2EDuration="2.680459522s" totalImagesPullingTime="1.158673437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:20.679455127 +0000 UTC m=+82.965872092" watchObservedRunningTime="2026-09-10 22:59:20.680459522 +0000 UTC m=+82.966876468"
Sep 10 22:59:20.845523 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.845489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.845523 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:20.845523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:20.845748 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.845620    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:59:20.845748 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.845634    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:20.845748 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.845683    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:21.845670068 +0000 UTC m=+84.132086999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:20.845748 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:20.845694    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:21.845688991 +0000 UTC m=+84.132105916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : configmap "kube-rbac-proxy" not found
Sep 10 22:59:21.249061 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:21.249024    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:21.249241 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.249181    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:21.249284 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.249246    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:29.249230939 +0000 UTC m=+91.535647866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:21.853878 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:21.853832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:21.853878 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:21.853878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:21.854399 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.853948    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:59:21.854399 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.854017    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:23.853997993 +0000 UTC m=+86.140414921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : configmap "kube-rbac-proxy" not found
Sep 10 22:59:21.854399 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.854026    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:21.854399 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:21.854064    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:23.854054169 +0000 UTC m=+86.140471093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:22.751986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:22.751957    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5d2lx_81390241-df7f-40c8-a3a2-25e9a1cdd7de/dns-node-resolver/0.log"
Sep 10 22:59:23.869190 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:23.869151    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:23.869190 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:23.869189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:23.869665 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:23.869286    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:59:23.869665 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:23.869320    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:23.869665 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:23.869367    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:27.869352971 +0000 UTC m=+90.155769909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:23.869665 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:23.869381    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:27.869374348 +0000 UTC m=+90.155791272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : configmap "kube-rbac-proxy" not found
Sep 10 22:59:23.955003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:23.954975    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rvc4_8284fd76-acaf-4918-a0b3-cb7b06791f81/node-ca/0.log"
Sep 10 22:59:24.552003 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:24.551979    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jnbw5_7a07f79f-2c5b-44dd-b7c9-556bfe289520/serve-healthcheck-canary/0.log"
Sep 10 22:59:25.152771 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:25.152717    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wz6k9_1965f0ec-a915-45e0-b3c4-830431034b52/migrator/0.log"
Sep 10 22:59:25.355099 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:25.355073    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wz6k9_1965f0ec-a915-45e0-b3c4-830431034b52/graceful-termination/0.log"
Sep 10 22:59:27.902449 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:27.902412    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:27.902449 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:27.902449    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:27.902896 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:27.902535    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:59:27.902896 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:27.902553    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:27.902896 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:27.902597    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:35.902582588 +0000 UTC m=+98.188999548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:59:27.902896 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:27.902614    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm podName:4edd12a4-1332-449e-9c82-b9f8b63002a2 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:35.902607923 +0000 UTC m=+98.189024847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n5mf" (UID: "4edd12a4-1332-449e-9c82-b9f8b63002a2") : configmap "kube-rbac-proxy" not found
Sep 10 22:59:29.312921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:29.312882    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:29.313346 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:29.313012    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:29.313346 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:29.313063    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls podName:21d6b7ee-8dba-4987-996d-38cd4cf1f37a nodeName:}" failed. No retries permitted until 2026-09-10 22:59:45.313050056 +0000 UTC m=+107.599466981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-nh5gm" (UID: "21d6b7ee-8dba-4987-996d-38cd4cf1f37a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 22:59:35.968672 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.968615    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:35.968672 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.968673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:35.969314 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.968691    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:35.969314 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.968725    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:35.969314 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.969203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4edd12a4-1332-449e-9c82-b9f8b63002a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:35.971168 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.971144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c0bad51-63d6-45a8-9ce2-dab24b009d0d-metrics-tls\") pod \"dns-default-fsc8h\" (UID: \"5c0bad51-63d6-45a8-9ce2-dab24b009d0d\") " pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:35.971267 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.971164    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4edd12a4-1332-449e-9c82-b9f8b63002a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n5mf\" (UID: \"4edd12a4-1332-449e-9c82-b9f8b63002a2\") " pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:35.971379 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:35.971357    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"image-registry-6bd556656-6vztw\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") " pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:36.012230 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.012203    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vnj2j\""
Sep 10 22:59:36.020325 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.020284    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:36.026045 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.026027    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-677w4\""
Sep 10 22:59:36.034153 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.034118    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:36.146410 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.146353    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8n5mf"
Sep 10 22:59:36.304702 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.304576    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 22:59:36.308331 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:36.308262    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111cbf4e_9628_44f6_b617_a46a60b66c0c.slice/crio-81275a55e01e761ecc7bffae21ade4c393aea4af10b1c8992dc8d675bc34ae44 WatchSource:0}: Error finding container 81275a55e01e761ecc7bffae21ade4c393aea4af10b1c8992dc8d675bc34ae44: Status 404 returned error can't find the container with id 81275a55e01e761ecc7bffae21ade4c393aea4af10b1c8992dc8d675bc34ae44
Sep 10 22:59:36.344998 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.344970    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fsc8h"]
Sep 10 22:59:36.349633 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:36.349580    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0bad51_63d6_45a8_9ce2_dab24b009d0d.slice/crio-bdf75c3168a380647f6c49c10a2c92e312475529b8585e8aae4b522d5ddf00ce WatchSource:0}: Error finding container bdf75c3168a380647f6c49c10a2c92e312475529b8585e8aae4b522d5ddf00ce: Status 404 returned error can't find the container with id bdf75c3168a380647f6c49c10a2c92e312475529b8585e8aae4b522d5ddf00ce
Sep 10 22:59:36.416583 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.416553    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8n5mf"]
Sep 10 22:59:36.419673 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:36.419645    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4edd12a4_1332_449e_9c82_b9f8b63002a2.slice/crio-67c5c25e85b87b3f3c47f6402f7c12ea96c4a3ecfbe739cd9f0eba5a29d83f01 WatchSource:0}: Error finding container 67c5c25e85b87b3f3c47f6402f7c12ea96c4a3ecfbe739cd9f0eba5a29d83f01: Status 404 returned error can't find the container with id 67c5c25e85b87b3f3c47f6402f7c12ea96c4a3ecfbe739cd9f0eba5a29d83f01
Sep 10 22:59:36.715147 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.715100    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n5mf" event={"ID":"4edd12a4-1332-449e-9c82-b9f8b63002a2","Type":"ContainerStarted","Data":"161c1f8d01827d5d255d7af0617bb671783b8e0fb4e6df55823724c9d8b5f934"}
Sep 10 22:59:36.715334 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.715177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n5mf" event={"ID":"4edd12a4-1332-449e-9c82-b9f8b63002a2","Type":"ContainerStarted","Data":"67c5c25e85b87b3f3c47f6402f7c12ea96c4a3ecfbe739cd9f0eba5a29d83f01"}
Sep 10 22:59:36.716565 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.716534    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fsc8h" event={"ID":"5c0bad51-63d6-45a8-9ce2-dab24b009d0d","Type":"ContainerStarted","Data":"bdf75c3168a380647f6c49c10a2c92e312475529b8585e8aae4b522d5ddf00ce"}
Sep 10 22:59:36.718071 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.718048    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd556656-6vztw" event={"ID":"111cbf4e-9628-44f6-b617-a46a60b66c0c","Type":"ContainerStarted","Data":"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"}
Sep 10 22:59:36.718170 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.718085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd556656-6vztw" event={"ID":"111cbf4e-9628-44f6-b617-a46a60b66c0c","Type":"ContainerStarted","Data":"81275a55e01e761ecc7bffae21ade4c393aea4af10b1c8992dc8d675bc34ae44"}
Sep 10 22:59:36.718170 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.718114    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:36.740740 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:36.740692    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bd556656-6vztw" podStartSLOduration=98.740674671 podStartE2EDuration="1m38.740674671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:36.739690535 +0000 UTC m=+99.026107482" watchObservedRunningTime="2026-09-10 22:59:36.740674671 +0000 UTC m=+99.027091618"
Sep 10 22:59:37.723936 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:37.723896    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n5mf" event={"ID":"4edd12a4-1332-449e-9c82-b9f8b63002a2","Type":"ContainerStarted","Data":"ed81bfc527057955c145b230ea2f873f3f4ef7087d42a7611c864b2d3ff678f8"}
Sep 10 22:59:38.730204 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:38.730161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fsc8h" event={"ID":"5c0bad51-63d6-45a8-9ce2-dab24b009d0d","Type":"ContainerStarted","Data":"0094fa4a94ed1860749433a107f7817757bbf7b826d120cb42ad2df7f73baf75"}
Sep 10 22:59:38.730648 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:38.730270    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fsc8h" event={"ID":"5c0bad51-63d6-45a8-9ce2-dab24b009d0d","Type":"ContainerStarted","Data":"987046edf34bc8d0c3865a8d2ca2d9e9ff18214d25dbccb53d7ce370cff9b90a"}
Sep 10 22:59:38.730648 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:38.730347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:38.757415 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:38.757361    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fsc8h" podStartSLOduration=65.093821938 podStartE2EDuration="1m6.757342794s" totalImagesPullingTime="1.663520856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:38.756639466 +0000 UTC m=+101.043056414" watchObservedRunningTime="2026-09-10 22:59:38.757342794 +0000 UTC m=+101.043759740"
Sep 10 22:59:39.735344 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:39.735306    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n5mf" event={"ID":"4edd12a4-1332-449e-9c82-b9f8b63002a2","Type":"ContainerStarted","Data":"42dbc3de8e62e45dc52703ca169aa9c5c4d9e82d252c87f32fc49fdb6207c33f"}
Sep 10 22:59:40.623469 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:40.623438    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ld8ds"
Sep 10 22:59:40.643279 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:40.643226    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8n5mf" podStartSLOduration=18.296596191 podStartE2EDuration="20.643208682s" totalImagesPullingTime="2.346612491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:59:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:39.753717406 +0000 UTC m=+102.040134352" watchObservedRunningTime="2026-09-10 22:59:40.643208682 +0000 UTC m=+102.929625629"
Sep 10 22:59:45.337437 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:45.337380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:45.339949 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:45.339927    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21d6b7ee-8dba-4987-996d-38cd4cf1f37a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-nh5gm\" (UID: \"21d6b7ee-8dba-4987-996d-38cd4cf1f37a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:45.531067 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:45.531026    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"
Sep 10 22:59:45.802118 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:45.802089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm"]
Sep 10 22:59:45.804940 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:45.804911    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d6b7ee_8dba_4987_996d_38cd4cf1f37a.slice/crio-41bd41dc7d823f6d087c17a4e11e5373bfc36af37f61c64edf0a3eb812d84268 WatchSource:0}: Error finding container 41bd41dc7d823f6d087c17a4e11e5373bfc36af37f61c64edf0a3eb812d84268: Status 404 returned error can't find the container with id 41bd41dc7d823f6d087c17a4e11e5373bfc36af37f61c64edf0a3eb812d84268
Sep 10 22:59:46.758814 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:46.758776    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm" event={"ID":"21d6b7ee-8dba-4987-996d-38cd4cf1f37a","Type":"ContainerStarted","Data":"41bd41dc7d823f6d087c17a4e11e5373bfc36af37f61c64edf0a3eb812d84268"}
Sep 10 22:59:47.764159 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.764118    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm" event={"ID":"21d6b7ee-8dba-4987-996d-38cd4cf1f37a","Type":"ContainerStarted","Data":"d620e246cc41ab74ba762f4a9d23a5fa80da1515c28f7f1cdde16b7dcfea55ce"}
Sep 10 22:59:47.781525 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.781475    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-nh5gm" podStartSLOduration=33.20394236 podStartE2EDuration="34.781456383s" totalImagesPullingTime="1.577514023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:47.780907238 +0000 UTC m=+110.067324210" watchObservedRunningTime="2026-09-10 22:59:47.781456383 +0000 UTC m=+110.067873331"
Sep 10 22:59:47.905532 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.905497    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"]
Sep 10 22:59:47.907555 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.907535    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:47.909878 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.909856    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 22:59:47.909878 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.909864    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4jgmh\""
Sep 10 22:59:47.917748 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.917721    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"]
Sep 10 22:59:47.956709 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:47.956672    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hhmgm\" (UID: \"4c8d9209-6d77-4212-acb7-85e58a28f0d4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:48.058076 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:48.057978    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hhmgm\" (UID: \"4c8d9209-6d77-4212-acb7-85e58a28f0d4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:48.058256 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:48.058106    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 22:59:48.058256 ip-10-0-137-120 kubenswrapper[2565]: E0910 22:59:48.058174    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates podName:4c8d9209-6d77-4212-acb7-85e58a28f0d4 nodeName:}" failed. No retries permitted until 2026-09-10 22:59:48.558154878 +0000 UTC m=+110.844571809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-hhmgm" (UID: "4c8d9209-6d77-4212-acb7-85e58a28f0d4") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 10 22:59:48.562866 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:48.562825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hhmgm\" (UID: \"4c8d9209-6d77-4212-acb7-85e58a28f0d4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:48.565434 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:48.565407    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4c8d9209-6d77-4212-acb7-85e58a28f0d4-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-hhmgm\" (UID: \"4c8d9209-6d77-4212-acb7-85e58a28f0d4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:48.737786 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:48.737755    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fsc8h"
Sep 10 22:59:48.819203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:48.819122    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:49.097646 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:49.097569    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"]
Sep 10 22:59:49.100438 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:49.100410    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8d9209_6d77_4212_acb7_85e58a28f0d4.slice/crio-ec6f499c2bafee3d6f94d1d33c574a765a996649eb8b3a43f09a0521a6edace8 WatchSource:0}: Error finding container ec6f499c2bafee3d6f94d1d33c574a765a996649eb8b3a43f09a0521a6edace8: Status 404 returned error can't find the container with id ec6f499c2bafee3d6f94d1d33c574a765a996649eb8b3a43f09a0521a6edace8
Sep 10 22:59:49.773386 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:49.773345    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm" event={"ID":"4c8d9209-6d77-4212-acb7-85e58a28f0d4","Type":"ContainerStarted","Data":"ec6f499c2bafee3d6f94d1d33c574a765a996649eb8b3a43f09a0521a6edace8"}
Sep 10 22:59:50.777943 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:50.777910    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm" event={"ID":"4c8d9209-6d77-4212-acb7-85e58a28f0d4","Type":"ContainerStarted","Data":"6c7d916098aeb41363056b084a94987ac0a2d4bb212965c95ee14ac015f727b9"}
Sep 10 22:59:50.778438 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:50.778120    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:50.782844 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:50.782813    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm"
Sep 10 22:59:50.810996 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:50.810945    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-hhmgm" podStartSLOduration=2.685184257 podStartE2EDuration="3.810928214s" totalImagesPullingTime="1.125743957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:50.809556051 +0000 UTC m=+113.095973009" watchObservedRunningTime="2026-09-10 22:59:50.810928214 +0000 UTC m=+113.097345190"
Sep 10 22:59:55.387746 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.387717    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t6q2j"]
Sep 10 22:59:55.390198 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.390179    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.392400 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.392378    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 22:59:55.392531 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.392377    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-22fhz\""
Sep 10 22:59:55.392531 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.392377    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 22:59:55.392953 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.392928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 22:59:55.393092 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.392987    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 22:59:55.513503 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-root\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-metrics-client-ca\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-textfile\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-tls\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513669    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-wtmp\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-sys\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513713    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgtzn\" (UniqueName: \"kubernetes.io/projected/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-kube-api-access-rgtzn\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513738    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.513783 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.513766    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615028 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.614973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-wtmp\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615028 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615030    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-sys\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615028 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615047    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgtzn\" (UniqueName: \"kubernetes.io/projected/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-kube-api-access-rgtzn\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-root\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615113    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-sys\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-metrics-client-ca\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-wtmp\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-textfile\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615215    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-tls\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615381 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-root\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615698 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615596    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-textfile\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615790 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615772    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-metrics-client-ca\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.615827 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.615815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.618383 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.618361    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.618451 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.618361    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-node-exporter-tls\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.625353 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.625329    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgtzn\" (UniqueName: \"kubernetes.io/projected/b23b5603-6431-4e7e-bbdf-e1c5246f4c7e-kube-api-access-rgtzn\") pod \"node-exporter-t6q2j\" (UID: \"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e\") " pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.699818 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.699783    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t6q2j"
Sep 10 22:59:55.710485 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:55.710457    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb23b5603_6431_4e7e_bbdf_e1c5246f4c7e.slice/crio-a1ca9876a222d134ae737fcdc78024295b2d2d1c76d4c4270be6585eccbd9f28 WatchSource:0}: Error finding container a1ca9876a222d134ae737fcdc78024295b2d2d1c76d4c4270be6585eccbd9f28: Status 404 returned error can't find the container with id a1ca9876a222d134ae737fcdc78024295b2d2d1c76d4c4270be6585eccbd9f28
Sep 10 22:59:55.794184 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:55.794141    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6q2j" event={"ID":"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e","Type":"ContainerStarted","Data":"a1ca9876a222d134ae737fcdc78024295b2d2d1c76d4c4270be6585eccbd9f28"}
Sep 10 22:59:56.024486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.024408    2565 patch_prober.go:28] interesting pod/image-registry-6bd556656-6vztw container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 22:59:56.024486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.024463    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6bd556656-6vztw" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:59:56.462223 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.462181    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 22:59:56.464793 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.464766    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.466870 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.466846    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 22:59:56.466986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.466900    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-khtx8\""
Sep 10 22:59:56.467203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467179    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 22:59:56.467203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467195    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 22:59:56.467203 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467182    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 22:59:56.467525 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467269    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 22:59:56.467755 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467737    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 22:59:56.467863 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467761    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 22:59:56.467863 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467809    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 22:59:56.467863 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.467761    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 22:59:56.482690 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.482596    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 22:59:56.624692 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.624692 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.624928 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624786    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqdc9\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.624928 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624886    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.624928 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625046 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.624955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625046 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625003    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625046 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625046 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625039    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625192 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625089    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625192 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625104    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625192 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.625192 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.625172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726255 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726221    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726419 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726260    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqdc9\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726419 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726419 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726419 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726419 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726430    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726596    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.726670 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.726619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.729986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.727355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.729986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.727609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.729986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.727731    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.729986 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.729517    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.730895 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.730841    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.731020 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.730975    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.731115 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.731094    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.731179 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.731111    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.731471 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.731274    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.732176 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.732153    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.732382 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.732367    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.732527 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.732513    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.735180 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.735156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqdc9\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9\") pod \"alertmanager-main-0\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.775570 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.775536    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 22:59:56.800465 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:56.800422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6q2j" event={"ID":"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e","Type":"ContainerStarted","Data":"c52dde3c8f2f50f6700ebe0253a166b0ea47e2e7756dbe71fc11011f5dfbf061"}
Sep 10 22:59:57.064648 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.064560    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 22:59:57.067124 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:57.067101    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50054a1f_d461_4d8b_8c30_350136ea9b14.slice/crio-dba9b3d4816ebdd10b7c8aa503f21d828fe3c9de3d718c22e4f1a63320911ab7 WatchSource:0}: Error finding container dba9b3d4816ebdd10b7c8aa503f21d828fe3c9de3d718c22e4f1a63320911ab7: Status 404 returned error can't find the container with id dba9b3d4816ebdd10b7c8aa503f21d828fe3c9de3d718c22e4f1a63320911ab7
Sep 10 22:59:57.364828 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.364741    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"]
Sep 10 22:59:57.367558 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.367541    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.369921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.369897    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 22:59:57.369921 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.369917    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 22:59:57.370087 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.369937    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 22:59:57.370087 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.370025    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dm969\""
Sep 10 22:59:57.370400 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.370382    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 22:59:57.370472 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.370459    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3bpacms9jpvs8\""
Sep 10 22:59:57.370644 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.370626    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 22:59:57.383592 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.383568    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"]
Sep 10 22:59:57.535494 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535494 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2ft\" (UniqueName: \"kubernetes.io/projected/1c3e48b1-48cd-454f-a128-14eb8486b892-kube-api-access-hw2ft\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535707    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535741    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-grpc-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.535950 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.535802    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3e48b1-48cd-454f-a128-14eb8486b892-metrics-client-ca\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.636635 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.636635 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.636853 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636797    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.636915 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2ft\" (UniqueName: \"kubernetes.io/projected/1c3e48b1-48cd-454f-a128-14eb8486b892-kube-api-access-hw2ft\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.636915 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.637011 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636944    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-grpc-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.637011 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.636977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3e48b1-48cd-454f-a128-14eb8486b892-metrics-client-ca\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.637319 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.637233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.638486 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.638457    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c3e48b1-48cd-454f-a128-14eb8486b892-metrics-client-ca\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.639210 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.639182    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.639337 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.639320    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.639759 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.639735    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.639847 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.639805    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.640071 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.640033    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.640157 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.640074    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1c3e48b1-48cd-454f-a128-14eb8486b892-secret-grpc-tls\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.647204 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.647179    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2ft\" (UniqueName: \"kubernetes.io/projected/1c3e48b1-48cd-454f-a128-14eb8486b892-kube-api-access-hw2ft\") pod \"thanos-querier-8555fcfb8-m6dmm\" (UID: \"1c3e48b1-48cd-454f-a128-14eb8486b892\") " pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.676302 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.676250    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 22:59:57.730067 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.729882    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 22:59:57.805847 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.805801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"dba9b3d4816ebdd10b7c8aa503f21d828fe3c9de3d718c22e4f1a63320911ab7"}
Sep 10 22:59:57.807669 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.807636    2565 generic.go:356] "Generic (PLEG): container finished" podID="b23b5603-6431-4e7e-bbdf-e1c5246f4c7e" containerID="c52dde3c8f2f50f6700ebe0253a166b0ea47e2e7756dbe71fc11011f5dfbf061" exitCode=0
Sep 10 22:59:57.807818 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.807765    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6q2j" event={"ID":"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e","Type":"ContainerDied","Data":"c52dde3c8f2f50f6700ebe0253a166b0ea47e2e7756dbe71fc11011f5dfbf061"}
Sep 10 22:59:57.960728 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:57.960700    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"]
Sep 10 22:59:57.963504 ip-10-0-137-120 kubenswrapper[2565]: W0910 22:59:57.963473    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3e48b1_48cd_454f_a128_14eb8486b892.slice/crio-17c5537e0d98bfe8821128e358ae2855965ab1e4e8fafd7e8a6f1c12d3468835 WatchSource:0}: Error finding container 17c5537e0d98bfe8821128e358ae2855965ab1e4e8fafd7e8a6f1c12d3468835: Status 404 returned error can't find the container with id 17c5537e0d98bfe8821128e358ae2855965ab1e4e8fafd7e8a6f1c12d3468835
Sep 10 22:59:58.814973 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.814905    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5" exitCode=0
Sep 10 22:59:58.815628 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.815016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"}
Sep 10 22:59:58.818817 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.818770    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"17c5537e0d98bfe8821128e358ae2855965ab1e4e8fafd7e8a6f1c12d3468835"}
Sep 10 22:59:58.822147 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.822117    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6q2j" event={"ID":"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e","Type":"ContainerStarted","Data":"67ca2739666cd0a7c96e58a80d05cdbf8a6cf5749f7779c9eef85ccccc46106a"}
Sep 10 22:59:58.822271 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.822174    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6q2j" event={"ID":"b23b5603-6431-4e7e-bbdf-e1c5246f4c7e","Type":"ContainerStarted","Data":"b024bf06b5fe093bd8bb15947a89249d66be904d7f8a3e148efe3dcc3a9f4ec7"}
Sep 10 22:59:58.879660 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:58.879601    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t6q2j" podStartSLOduration=2.814790511 podStartE2EDuration="3.879581328s" totalImagesPullingTime="977.438961ms" totalInitContainerRuntime="87.351856ms" isStatefulPod=true podCreationTimestamp="2026-09-10 22:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:59:58.879256777 +0000 UTC m=+121.165673749" watchObservedRunningTime="2026-09-10 22:59:58.879581328 +0000 UTC m=+121.165998275"
Sep 10 22:59:59.804033 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.803985    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-59f4745cc9-zjvrn"]
Sep 10 22:59:59.807643 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.807621    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.810190 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.809945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-25ssd\""
Sep 10 22:59:59.810190 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.809994    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 22:59:59.810822 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.810603    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 22:59:59.810822 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.810615    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 22:59:59.810822 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.810699    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 22:59:59.810822 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.810648    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b39pmfasppqjl\""
Sep 10 22:59:59.817945 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.817923    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59f4745cc9-zjvrn"]
Sep 10 22:59:59.956105 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956074    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-client-certs\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956105 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-tls\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9th\" (UniqueName: \"kubernetes.io/projected/59bdf5f2-de56-416e-b66c-64a1744fce59-kube-api-access-cn9th\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956153    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-metrics-server-audit-profiles\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-client-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956345 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956306    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/59bdf5f2-de56-416e-b66c-64a1744fce59-audit-log\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 22:59:59.956507 ip-10-0-137-120 kubenswrapper[2565]: I0910 22:59:59.956394    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057700 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-metrics-server-audit-profiles\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057700 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-client-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/59bdf5f2-de56-416e-b66c-64a1744fce59-audit-log\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-client-certs\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-tls\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.057925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.057875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9th\" (UniqueName: \"kubernetes.io/projected/59bdf5f2-de56-416e-b66c-64a1744fce59-kube-api-access-cn9th\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.058149 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.058132    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/59bdf5f2-de56-416e-b66c-64a1744fce59-audit-log\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.058750 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.058725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.058865 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.058725    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/59bdf5f2-de56-416e-b66c-64a1744fce59-metrics-server-audit-profiles\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.060867 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.060842    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-client-certs\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.061042 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.061025    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-secret-metrics-server-tls\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.061389 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.061368    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59bdf5f2-de56-416e-b66c-64a1744fce59-client-ca-bundle\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.076477 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.076449    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9th\" (UniqueName: \"kubernetes.io/projected/59bdf5f2-de56-416e-b66c-64a1744fce59-kube-api-access-cn9th\") pod \"metrics-server-59f4745cc9-zjvrn\" (UID: \"59bdf5f2-de56-416e-b66c-64a1744fce59\") " pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.120209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.120165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:00.145232 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.145195    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"]
Sep 10 23:00:00.148748 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.148725    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.152030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.151980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 23:00:00.152199 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.151986    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bl7qg\""
Sep 10 23:00:00.160327 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.160284    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"]
Sep 10 23:00:00.259075 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.259042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpx4\" (UniqueName: \"kubernetes.io/projected/dd44ba6c-9080-47a6-b1db-39469c41a613-kube-api-access-vcpx4\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.259275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.259106    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dd44ba6c-9080-47a6-b1db-39469c41a613-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.359810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.359784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dd44ba6c-9080-47a6-b1db-39469c41a613-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.359951 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.359931    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpx4\" (UniqueName: \"kubernetes.io/projected/dd44ba6c-9080-47a6-b1db-39469c41a613-kube-api-access-vcpx4\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.362369 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.362345    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dd44ba6c-9080-47a6-b1db-39469c41a613-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.372364 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.372342    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpx4\" (UniqueName: \"kubernetes.io/projected/dd44ba6c-9080-47a6-b1db-39469c41a613-kube-api-access-vcpx4\") pod \"monitoring-plugin-5d85778b9d-ch4xn\" (UID: \"dd44ba6c-9080-47a6-b1db-39469c41a613\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.461536 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.461172    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:00.630903 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.630876    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59f4745cc9-zjvrn"]
Sep 10 23:00:00.635189 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:00:00.635166    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59bdf5f2_de56_416e_b66c_64a1744fce59.slice/crio-544f9007634a925f718a6b789f5ded5e70a1f09f3c04f7facc5c1138d2843e46 WatchSource:0}: Error finding container 544f9007634a925f718a6b789f5ded5e70a1f09f3c04f7facc5c1138d2843e46: Status 404 returned error can't find the container with id 544f9007634a925f718a6b789f5ded5e70a1f09f3c04f7facc5c1138d2843e46
Sep 10 23:00:00.757103 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.757065    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"]
Sep 10 23:00:00.759704 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:00:00.759677    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd44ba6c_9080_47a6_b1db_39469c41a613.slice/crio-5125ce5dd54a49ddfeb8704762b34f24e92f4734e536f91103a53fb98ff89d3c WatchSource:0}: Error finding container 5125ce5dd54a49ddfeb8704762b34f24e92f4734e536f91103a53fb98ff89d3c: Status 404 returned error can't find the container with id 5125ce5dd54a49ddfeb8704762b34f24e92f4734e536f91103a53fb98ff89d3c
Sep 10 23:00:00.834815 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.834778    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"}
Sep 10 23:00:00.835209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.834861    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"}
Sep 10 23:00:00.835209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.834897    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"}
Sep 10 23:00:00.835209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.834927    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"}
Sep 10 23:00:00.835209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.834951    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"}
Sep 10 23:00:00.837016 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.836987    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"42480cd9e7ef2e992d486e7c859479de0429685bb16b2e73eb6c7d27c0c7aa81"}
Sep 10 23:00:00.837120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.837068    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"9d347183958eac95a6bfabd8697194353d28fa08cb2a3e200496c8e1ba2138c1"}
Sep 10 23:00:00.837120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.837109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"ead9b1ae7063ebf896b131dbadae8ab26e25bd9457ebc143dc59dbff8311c2f9"}
Sep 10 23:00:00.838072 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.838052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn" event={"ID":"dd44ba6c-9080-47a6-b1db-39469c41a613","Type":"ContainerStarted","Data":"5125ce5dd54a49ddfeb8704762b34f24e92f4734e536f91103a53fb98ff89d3c"}
Sep 10 23:00:00.839107 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.839084    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn" event={"ID":"59bdf5f2-de56-416e-b66c-64a1744fce59","Type":"ContainerStarted","Data":"544f9007634a925f718a6b789f5ded5e70a1f09f3c04f7facc5c1138d2843e46"}
Sep 10 23:00:00.862834 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:00.862763    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 23:00:01.694969 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.694947    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:00:01.699657 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.699637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.702560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.702535    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 23:00:01.702645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.702563    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 23:00:01.702645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.702635    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 23:00:01.702851 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.702834    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 23:00:01.702911 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.702866    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 23:00:01.703384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703060    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-am1ffhef871dr\""
Sep 10 23:00:01.703384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703066    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 23:00:01.703384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703248    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 23:00:01.703384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703360    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 23:00:01.703384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703378    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wq7xc\""
Sep 10 23:00:01.703687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703500    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 23:00:01.703687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703592    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 23:00:01.703788 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703743    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 23:00:01.704096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.703923    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 23:00:01.705484 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.705350    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 23:00:01.706384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.706364    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 23:00:01.714818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.714782    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:00:01.847135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.847106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerStarted","Data":"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"}
Sep 10 23:00:01.850769 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.850743    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"aa8021e6f6db1fb125b52af099e21bdc217fcf3e6ab1f24fa72a20646543f03a"}
Sep 10 23:00:01.850923 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.850814    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"94eb7b8799247fdc1e0d3c79cc397a3112a30d58fbb87401f2545648bfa0f818"}
Sep 10 23:00:01.850923 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.850862    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 23:00:01.850923 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.850898    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" event={"ID":"1c3e48b1-48cd-454f-a128-14eb8486b892","Type":"ContainerStarted","Data":"4b087e09adb4e452e009befb8c48b5cc3d46d9b7fe9544a13b89f6f026552464"}
Sep 10 23:00:01.872631 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872509    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.210643639 podStartE2EDuration="5.872489362s" totalImagesPullingTime="3.645581131s" totalInitContainerRuntime="16.264592ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:01.872090454 +0000 UTC m=+124.158507402" watchObservedRunningTime="2026-09-10 23:00:01.872489362 +0000 UTC m=+124.158906309"
Sep 10 23:00:01.872806 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872749    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872806 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872806 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872802    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872819    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872833    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872889    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872937    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872954    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.872976 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.872995    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873009    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lmfb\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873038    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873052    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873105    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873129    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.873363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.873144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.903685 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.903622    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm" podStartSLOduration=1.491878057 podStartE2EDuration="4.903598543s" totalImagesPullingTime="3.411720486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:01.901100103 +0000 UTC m=+124.187517061" watchObservedRunningTime="2026-09-10 23:00:01.903598543 +0000 UTC m=+124.190015491"
Sep 10 23:00:01.974128 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974088    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974168    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974343    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974366    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lmfb\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974470    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.974744 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.974556    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975036    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975527    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975756    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975796    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975815    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975914    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.976111 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.975963    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.977655 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.977629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.977756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.977665    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.977756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.977663    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.977756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.977696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.978886 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.978855    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.979013 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.978977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.979342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.979322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.979747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.979724    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.979900 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.979864    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.980496 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.980475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.983335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.981482    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.983335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.981948    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.983335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.982640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.983335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.982767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.984351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.983749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.986612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.986590    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:01.999224 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:01.999198    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lmfb\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb\") pod \"prometheus-k8s-0\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:02.035277 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.035232    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:02.484016 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.483927    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:00:02.488266 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:00:02.488231    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f9a188_8b65_43b5_a051_56263290f97a.slice/crio-3fe23dc72ad0a882f3aa9c7471d85619916fbb2da6680bba1ba2b7e62bed37bf WatchSource:0}: Error finding container 3fe23dc72ad0a882f3aa9c7471d85619916fbb2da6680bba1ba2b7e62bed37bf: Status 404 returned error can't find the container with id 3fe23dc72ad0a882f3aa9c7471d85619916fbb2da6680bba1ba2b7e62bed37bf
Sep 10 23:00:02.857186 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.857100    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74" exitCode=0
Sep 10 23:00:02.857628 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.857181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"}
Sep 10 23:00:02.857628 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.857252    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"3fe23dc72ad0a882f3aa9c7471d85619916fbb2da6680bba1ba2b7e62bed37bf"}
Sep 10 23:00:02.859013 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.858960    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn" event={"ID":"59bdf5f2-de56-416e-b66c-64a1744fce59","Type":"ContainerStarted","Data":"71a7811f3c0330f7925f5829eb9ec0d223f67d8b761efe08e625c52eef177a47"}
Sep 10 23:00:02.913847 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:02.913597    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn" podStartSLOduration=2.299032674 podStartE2EDuration="3.913580052s" totalImagesPullingTime="1.614547378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:02.912951107 +0000 UTC m=+125.199368056" watchObservedRunningTime="2026-09-10 23:00:02.913580052 +0000 UTC m=+125.199996999"
Sep 10 23:00:03.864308 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:03.864247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn" event={"ID":"dd44ba6c-9080-47a6-b1db-39469c41a613","Type":"ContainerStarted","Data":"29ab614c9d1159de785b5a21253c32ee60d676ff4b08f3e469d1c473b7c46721"}
Sep 10 23:00:03.864684 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:03.864496    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:03.869272 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:03.869242    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn"
Sep 10 23:00:03.882091 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:03.882047    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-ch4xn" podStartSLOduration=1.395455948 podStartE2EDuration="3.882032379s" totalImagesPullingTime="2.486576431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:03.880214407 +0000 UTC m=+126.166631355" watchObservedRunningTime="2026-09-10 23:00:03.882032379 +0000 UTC m=+126.168449325"
Sep 10 23:00:07.865048 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.865022    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8555fcfb8-m6dmm"
Sep 10 23:00:07.885369 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"}
Sep 10 23:00:07.885538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885400    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"}
Sep 10 23:00:07.885538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"}
Sep 10 23:00:07.885538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"}
Sep 10 23:00:07.885538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885476    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"}
Sep 10 23:00:07.885538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.885498    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerStarted","Data":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"}
Sep 10 23:00:07.935246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.935214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 23:00:07.938564 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.938537    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45d8b65e-7ea0-4ff9-83ad-6da69ade88a2-metrics-certs\") pod \"network-metrics-daemon-tvrpd\" (UID: \"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2\") " pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 23:00:07.950919 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:07.950847    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.718224165 podStartE2EDuration="6.950822499s" totalImagesPullingTime="4.210962903s" totalInitContainerRuntime="21.635431ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:00:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:07.946351145 +0000 UTC m=+130.232768092" watchObservedRunningTime="2026-09-10 23:00:07.950822499 +0000 UTC m=+130.237239447"
Sep 10 23:00:08.218465 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:08.218430    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hw9xb\""
Sep 10 23:00:08.226234 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:08.226210    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tvrpd"
Sep 10 23:00:08.498097 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:08.498075    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tvrpd"]
Sep 10 23:00:08.500038 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:00:08.500006    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d8b65e_7ea0_4ff9_83ad_6da69ade88a2.slice/crio-c110f2a9a4bfb7c910363c712dd5768a1e9c9dc87ec570143673159bfd453eed WatchSource:0}: Error finding container c110f2a9a4bfb7c910363c712dd5768a1e9c9dc87ec570143673159bfd453eed: Status 404 returned error can't find the container with id c110f2a9a4bfb7c910363c712dd5768a1e9c9dc87ec570143673159bfd453eed
Sep 10 23:00:08.890930 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:08.890835    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvrpd" event={"ID":"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2","Type":"ContainerStarted","Data":"c110f2a9a4bfb7c910363c712dd5768a1e9c9dc87ec570143673159bfd453eed"}
Sep 10 23:00:09.897077 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:09.896984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvrpd" event={"ID":"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2","Type":"ContainerStarted","Data":"2914e43c01d001e3c9d332142b6cd5a41a002553a44bf1d4882e3643f6b988db"}
Sep 10 23:00:09.897077 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:09.897051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tvrpd" event={"ID":"45d8b65e-7ea0-4ff9-83ad-6da69ade88a2","Type":"ContainerStarted","Data":"47e76f4366222e62f38b2a7edc1fe14432d8492ab29375ff576f10ce8c66da94"}
Sep 10 23:00:09.913905 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:09.913850    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tvrpd" podStartSLOduration=130.865195434 podStartE2EDuration="2m11.913832951s" totalImagesPullingTime="1.048637517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:00:09.912836084 +0000 UTC m=+132.199253062" watchObservedRunningTime="2026-09-10 23:00:09.913832951 +0000 UTC m=+132.200249898"
Sep 10 23:00:12.035926 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:12.035882    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:00:20.120689 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:20.120654    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:20.121169 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:20.120752    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:21.948219 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:21.948185    2565 generic.go:356] "Generic (PLEG): container finished" podID="11728049-5eea-493c-b523-c9b21283b07e" containerID="361a698ce215b030aa315a7732dbbd687ea4e6949802fa24a54679ead15a171c" exitCode=0
Sep 10 23:00:21.948637 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:21.948266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp" event={"ID":"11728049-5eea-493c-b523-c9b21283b07e","Type":"ContainerDied","Data":"361a698ce215b030aa315a7732dbbd687ea4e6949802fa24a54679ead15a171c"}
Sep 10 23:00:21.948679 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:21.948652    2565 scope.go:117] "RemoveContainer" containerID="361a698ce215b030aa315a7732dbbd687ea4e6949802fa24a54679ead15a171c"
Sep 10 23:00:22.954517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:22.954483    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-dvflp" event={"ID":"11728049-5eea-493c-b523-c9b21283b07e","Type":"ContainerStarted","Data":"4defbc2637010312c7f55980d955fdb7320148cc4c31daf3c3cda96c0c7a5a28"}
Sep 10 23:00:25.886955 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:25.886907    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6bd556656-6vztw" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerName="registry" containerID="cri-o://099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384" gracePeriod=30
Sep 10 23:00:26.130248 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.130221    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 23:00:26.201268 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201227    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201279    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hbng\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201318    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201334    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201470    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201541    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201594    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201681    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets\") pod \"111cbf4e-9628-44f6-b617-a46a60b66c0c\" (UID: \"111cbf4e-9628-44f6-b617-a46a60b66c0c\") "
Sep 10 23:00:26.201847 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201745    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:00:26.201902 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201753    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:00:26.201973 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201954    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-trusted-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.202030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.201975    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-certificates\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.209740 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.209655    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:00:26.210017 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.209972    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:00:26.210148 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.210105    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng" (OuterVolumeSpecName: "kube-api-access-2hbng") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "kube-api-access-2hbng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:00:26.210148 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.210110    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:00:26.210148 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.210124    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:00:26.212682 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.212657    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "111cbf4e-9628-44f6-b617-a46a60b66c0c" (UID: "111cbf4e-9628-44f6-b617-a46a60b66c0c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:00:26.303207 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303163    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-registry-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.303207 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303195    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-bound-sa-token\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.303207 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303210    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/111cbf4e-9628-44f6-b617-a46a60b66c0c-ca-trust-extracted\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.303484 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303223    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-image-registry-private-configuration\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.303484 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303236    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/111cbf4e-9628-44f6-b617-a46a60b66c0c-installation-pull-secrets\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.303484 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.303248    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hbng\" (UniqueName: \"kubernetes.io/projected/111cbf4e-9628-44f6-b617-a46a60b66c0c-kube-api-access-2hbng\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:00:26.973469 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.973430    2565 generic.go:356] "Generic (PLEG): container finished" podID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerID="099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384" exitCode=0
Sep 10 23:00:26.973877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.973490    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bd556656-6vztw"
Sep 10 23:00:26.973877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.973502    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd556656-6vztw" event={"ID":"111cbf4e-9628-44f6-b617-a46a60b66c0c","Type":"ContainerDied","Data":"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"}
Sep 10 23:00:26.973877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.973541    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bd556656-6vztw" event={"ID":"111cbf4e-9628-44f6-b617-a46a60b66c0c","Type":"ContainerDied","Data":"81275a55e01e761ecc7bffae21ade4c393aea4af10b1c8992dc8d675bc34ae44"}
Sep 10 23:00:26.973877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.973568    2565 scope.go:117] "RemoveContainer" containerID="099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"
Sep 10 23:00:26.982285 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.982268    2565 scope.go:117] "RemoveContainer" containerID="099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"
Sep 10 23:00:26.982572 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:00:26.982547    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384\": container with ID starting with 099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384 not found: ID does not exist" containerID="099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"
Sep 10 23:00:26.982646 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.982582    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384"} err="failed to get container status \"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384\": rpc error: code = NotFound desc = could not find container \"099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384\": container with ID starting with 099a5338b824ba6946177cee4071a86e65374206f1f6d06b546f571e82f29384 not found: ID does not exist"
Sep 10 23:00:26.992271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.992226    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 23:00:26.994061 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:26.994037    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bd556656-6vztw"]
Sep 10 23:00:28.209126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:28.209095    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" path="/var/lib/kubelet/pods/111cbf4e-9628-44f6-b617-a46a60b66c0c/volumes"
Sep 10 23:00:40.126980 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:40.126951    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:00:40.131230 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:00:40.131201    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-59f4745cc9-zjvrn"
Sep 10 23:01:02.035562 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:02.035529    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:02.055759 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:02.055732    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:02.139813 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:02.139781    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:15.851831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.851792    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:15.852359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852107    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="alertmanager" containerID="cri-o://ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac" gracePeriod=120
Sep 10 23:01:15.852359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852163    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-metric" containerID="cri-o://be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85" gracePeriod=120
Sep 10 23:01:15.852359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852210    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-web" containerID="cri-o://211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6" gracePeriod=120
Sep 10 23:01:15.852359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852231    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="prom-label-proxy" containerID="cri-o://f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40" gracePeriod=120
Sep 10 23:01:15.852359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852274    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="config-reloader" containerID="cri-o://feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4" gracePeriod=120
Sep 10 23:01:15.852622 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:15.852267    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy" containerID="cri-o://49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe" gracePeriod=120
Sep 10 23:01:16.182521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182492    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40" exitCode=0
Sep 10 23:01:16.182521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182516    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe" exitCode=0
Sep 10 23:01:16.182521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182522    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4" exitCode=0
Sep 10 23:01:16.182521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182527    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac" exitCode=0
Sep 10 23:01:16.182777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"}
Sep 10 23:01:16.182777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"}
Sep 10 23:01:16.182777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182648    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"}
Sep 10 23:01:16.182777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:16.182671    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"}
Sep 10 23:01:17.089098 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.089071    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.141748 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.141715    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.141933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.141773    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.141933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.141811    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.141933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.141850    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142119    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142184    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:17.142243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142314    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142518    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142546    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142575    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142613    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqdc9\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142639    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web\") pod \"50054a1f-d461-4d8b-8c30-350136ea9b14\" (UID: \"50054a1f-d461-4d8b-8c30-350136ea9b14\") "
Sep 10 23:01:17.142957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.142895    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.143463 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.143411    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:01:17.143779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.143749    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:17.145733 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.145699    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume" (OuterVolumeSpecName: "config-volume") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.146436 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.146394    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.146551 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.146496    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:01:17.147631 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.147602    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out" (OuterVolumeSpecName: "config-out") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:01:17.147960 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.147914    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9" (OuterVolumeSpecName: "kube-api-access-mqdc9") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "kube-api-access-mqdc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:01:17.148058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.147958    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.148164 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.148138    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.148473 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.148446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.152482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.152451    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.183196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.183147    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config" (OuterVolumeSpecName: "web-config") pod "50054a1f-d461-4d8b-8c30-350136ea9b14" (UID: "50054a1f-d461-4d8b-8c30-350136ea9b14"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:17.190331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190237    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85" exitCode=0
Sep 10 23:01:17.190331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190271    2565 generic.go:356] "Generic (PLEG): container finished" podID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerID="211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6" exitCode=0
Sep 10 23:01:17.190487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190352    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.190487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"}
Sep 10 23:01:17.190578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190524    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"}
Sep 10 23:01:17.190578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190550    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"50054a1f-d461-4d8b-8c30-350136ea9b14","Type":"ContainerDied","Data":"dba9b3d4816ebdd10b7c8aa503f21d828fe3c9de3d718c22e4f1a63320911ab7"}
Sep 10 23:01:17.190642 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.190583    2565 scope.go:117] "RemoveContainer" containerID="f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"
Sep 10 23:01:17.199419 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.199404    2565 scope.go:117] "RemoveContainer" containerID="be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"
Sep 10 23:01:17.210389 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.210370    2565 scope.go:117] "RemoveContainer" containerID="49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"
Sep 10 23:01:17.217713 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.217683    2565 scope.go:117] "RemoveContainer" containerID="211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"
Sep 10 23:01:17.221956 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.221934    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:17.226657 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.226336    2565 scope.go:117] "RemoveContainer" containerID="feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"
Sep 10 23:01:17.229611 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.229581    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:17.234634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.234617    2565 scope.go:117] "RemoveContainer" containerID="ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"
Sep 10 23:01:17.242434 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.242415    2565 scope.go:117] "RemoveContainer" containerID="8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"
Sep 10 23:01:17.243237 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243219    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243242    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-config-out\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243258    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-web-config\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243271    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50054a1f-d461-4d8b-8c30-350136ea9b14-metrics-client-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243305    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-main-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243321    2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/50054a1f-d461-4d8b-8c30-350136ea9b14-alertmanager-main-db\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243334    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqdc9\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-kube-api-access-mqdc9\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243348    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243361    2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-cluster-tls-config\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243373    2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-config-volume\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243387    2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50054a1f-d461-4d8b-8c30-350136ea9b14-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.243583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.243401    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50054a1f-d461-4d8b-8c30-350136ea9b14-tls-assets\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:17.250035 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.250016    2565 scope.go:117] "RemoveContainer" containerID="f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"
Sep 10 23:01:17.250384 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.250365    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40\": container with ID starting with f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40 not found: ID does not exist" containerID="f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"
Sep 10 23:01:17.250463 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.250394    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"} err="failed to get container status \"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40\": rpc error: code = NotFound desc = could not find container \"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40\": container with ID starting with f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40 not found: ID does not exist"
Sep 10 23:01:17.250463 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.250414    2565 scope.go:117] "RemoveContainer" containerID="be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"
Sep 10 23:01:17.250730 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.250707    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85\": container with ID starting with be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85 not found: ID does not exist" containerID="be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"
Sep 10 23:01:17.250786 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.250727    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"} err="failed to get container status \"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85\": rpc error: code = NotFound desc = could not find container \"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85\": container with ID starting with be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85 not found: ID does not exist"
Sep 10 23:01:17.250786 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.250745    2565 scope.go:117] "RemoveContainer" containerID="49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"
Sep 10 23:01:17.251007 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.250986    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe\": container with ID starting with 49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe not found: ID does not exist" containerID="49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"
Sep 10 23:01:17.251132 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251007    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"} err="failed to get container status \"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe\": rpc error: code = NotFound desc = could not find container \"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe\": container with ID starting with 49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe not found: ID does not exist"
Sep 10 23:01:17.251132 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251021    2565 scope.go:117] "RemoveContainer" containerID="211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"
Sep 10 23:01:17.251416 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.251393    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6\": container with ID starting with 211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6 not found: ID does not exist" containerID="211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"
Sep 10 23:01:17.251520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251422    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"} err="failed to get container status \"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6\": rpc error: code = NotFound desc = could not find container \"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6\": container with ID starting with 211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6 not found: ID does not exist"
Sep 10 23:01:17.251520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251443    2565 scope.go:117] "RemoveContainer" containerID="feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"
Sep 10 23:01:17.251864 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.251821    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4\": container with ID starting with feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4 not found: ID does not exist" containerID="feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"
Sep 10 23:01:17.251945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251870    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"} err="failed to get container status \"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4\": rpc error: code = NotFound desc = could not find container \"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4\": container with ID starting with feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4 not found: ID does not exist"
Sep 10 23:01:17.251945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.251893    2565 scope.go:117] "RemoveContainer" containerID="ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"
Sep 10 23:01:17.252178 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.252161    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac\": container with ID starting with ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac not found: ID does not exist" containerID="ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"
Sep 10 23:01:17.252219 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252186    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"} err="failed to get container status \"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac\": rpc error: code = NotFound desc = could not find container \"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac\": container with ID starting with ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac not found: ID does not exist"
Sep 10 23:01:17.252219 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252202    2565 scope.go:117] "RemoveContainer" containerID="8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"
Sep 10 23:01:17.252514 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:17.252496    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5\": container with ID starting with 8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5 not found: ID does not exist" containerID="8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"
Sep 10 23:01:17.252575 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252519    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"} err="failed to get container status \"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5\": rpc error: code = NotFound desc = could not find container \"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5\": container with ID starting with 8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5 not found: ID does not exist"
Sep 10 23:01:17.252575 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252534    2565 scope.go:117] "RemoveContainer" containerID="f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"
Sep 10 23:01:17.252789 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252766    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40"} err="failed to get container status \"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40\": rpc error: code = NotFound desc = could not find container \"f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40\": container with ID starting with f071b7fcd5c8eaed749ff66529e49f9d3226a7b7d13a3db19cea32a67ac61a40 not found: ID does not exist"
Sep 10 23:01:17.252837 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.252793    2565 scope.go:117] "RemoveContainer" containerID="be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"
Sep 10 23:01:17.253056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253034    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85"} err="failed to get container status \"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85\": rpc error: code = NotFound desc = could not find container \"be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85\": container with ID starting with be8a2b8fe04896a4d68f794dbdce908d924dbcb33c5a2c5e7a5d3084cf117f85 not found: ID does not exist"
Sep 10 23:01:17.253115 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253057    2565 scope.go:117] "RemoveContainer" containerID="49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"
Sep 10 23:01:17.253355 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253334    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe"} err="failed to get container status \"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe\": rpc error: code = NotFound desc = could not find container \"49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe\": container with ID starting with 49ff100c7e6877c3f0a12ca874fab89b12e75e715484c7c82c2dc171b7f3fffe not found: ID does not exist"
Sep 10 23:01:17.253409 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253358    2565 scope.go:117] "RemoveContainer" containerID="211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"
Sep 10 23:01:17.253611 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253594    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6"} err="failed to get container status \"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6\": rpc error: code = NotFound desc = could not find container \"211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6\": container with ID starting with 211499b73f4e6171da56fb94f8fef1e3969cd8ea0dddcaca40c7aa2fc97626c6 not found: ID does not exist"
Sep 10 23:01:17.253661 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253612    2565 scope.go:117] "RemoveContainer" containerID="feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"
Sep 10 23:01:17.253837 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253820    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4"} err="failed to get container status \"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4\": rpc error: code = NotFound desc = could not find container \"feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4\": container with ID starting with feb8a4514ef5665f030ad8a2d16e683caafa61f8830acc95ecaa3c2b5766fae4 not found: ID does not exist"
Sep 10 23:01:17.253883 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.253837    2565 scope.go:117] "RemoveContainer" containerID="ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"
Sep 10 23:01:17.254056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.254040    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac"} err="failed to get container status \"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac\": rpc error: code = NotFound desc = could not find container \"ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac\": container with ID starting with ec76175f27cac88702190820ad13d0d327a47496f839cf194ec42120390a7bac not found: ID does not exist"
Sep 10 23:01:17.254116 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.254056    2565 scope.go:117] "RemoveContainer" containerID="8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"
Sep 10 23:01:17.254250 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.254236    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5"} err="failed to get container status \"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5\": rpc error: code = NotFound desc = could not find container \"8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5\": container with ID starting with 8a01cd2b4a454ab5f95a8dc404ea1df4e4c6298f6e1918b0218126da973358e5 not found: ID does not exist"
Sep 10 23:01:17.268205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268175    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:17.268575 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268557    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerName="registry"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268577    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerName="registry"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268590    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-metric"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268598    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-metric"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268610    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="init-config-reloader"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268619    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="init-config-reloader"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268635    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-web"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268643    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-web"
Sep 10 23:01:17.268656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268657    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="config-reloader"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268666    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="config-reloader"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268678    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="alertmanager"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268687    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="alertmanager"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268706    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268715    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268732    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="prom-label-proxy"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268740    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="prom-label-proxy"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268820    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="111cbf4e-9628-44f6-b617-a46a60b66c0c" containerName="registry"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268834    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="alertmanager"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268843    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-web"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268852    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy-metric"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268861    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="kube-rbac-proxy"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268869    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="config-reloader"
Sep 10 23:01:17.269027 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.268878    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" containerName="prom-label-proxy"
Sep 10 23:01:17.272993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.272970    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.275412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275387    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 23:01:17.275544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275419    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 23:01:17.275544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275457    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 23:01:17.275544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275394    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 23:01:17.275697 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275638    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-khtx8\""
Sep 10 23:01:17.275741 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275695    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 23:01:17.275787 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275740    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 23:01:17.275838 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275821    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 23:01:17.275917 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.275900    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 23:01:17.281452 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.281431    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 23:01:17.288309 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.288273    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:17.344129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344134    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344151    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-web-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344171    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344187    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344347    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-out\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344576 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344576 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344455    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344576 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344524    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344576 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344571    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.344696 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.344592    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq7jd\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-kube-api-access-dq7jd\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445360 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445223    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-out\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445360 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445360 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445320    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445360 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445397    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445447    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445478    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq7jd\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-kube-api-access-dq7jd\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445529    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-web-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445578    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.445688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.445649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.446238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.446215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.446594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.446259    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.446594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.446558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.448792 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.448739    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.448950 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.448920    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-web-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449064 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.448983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449064 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.448999    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-out\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449182 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.449117    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449317 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.449275    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449375 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.449343    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-config-volume\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.449790 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.449771    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.450422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.450406    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5f858376-5b6b-4da1-b0b7-a19f271f4708-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.455990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.455963    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq7jd\" (UniqueName: \"kubernetes.io/projected/5f858376-5b6b-4da1-b0b7-a19f271f4708-kube-api-access-dq7jd\") pod \"alertmanager-main-0\" (UID: \"5f858376-5b6b-4da1-b0b7-a19f271f4708\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.583018 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.582978    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 23:01:17.866268 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:17.866245    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 23:01:17.868326 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:01:17.868271    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f858376_5b6b_4da1_b0b7_a19f271f4708.slice/crio-a9913197441191af52c4e80a20d299deeca0f7b68e026eecd9919f395b825d75 WatchSource:0}: Error finding container a9913197441191af52c4e80a20d299deeca0f7b68e026eecd9919f395b825d75: Status 404 returned error can't find the container with id a9913197441191af52c4e80a20d299deeca0f7b68e026eecd9919f395b825d75
Sep 10 23:01:18.196204 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:18.196161    2565 generic.go:356] "Generic (PLEG): container finished" podID="5f858376-5b6b-4da1-b0b7-a19f271f4708" containerID="b4e54e4b05dbd7275edb7396f5fe15d9400f31477926ca15f4191d11cc3b9367" exitCode=0
Sep 10 23:01:18.196690 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:18.196246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerDied","Data":"b4e54e4b05dbd7275edb7396f5fe15d9400f31477926ca15f4191d11cc3b9367"}
Sep 10 23:01:18.196690 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:18.196338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"a9913197441191af52c4e80a20d299deeca0f7b68e026eecd9919f395b825d75"}
Sep 10 23:01:18.209228 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:18.209199    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50054a1f-d461-4d8b-8c30-350136ea9b14" path="/var/lib/kubelet/pods/50054a1f-d461-4d8b-8c30-350136ea9b14/volumes"
Sep 10 23:01:19.204544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204505    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"d64b2063695b96cc0c961a50e875c78689be54280a1cbd6f9ce127c4c8201a36"}
Sep 10 23:01:19.205130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204620    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"1c669d224a320f70152f905d5c2f272aaf5e401f0f5a767d03e5162c697d2397"}
Sep 10 23:01:19.205130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204662    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"2be8c9ac172a110730112bdcbcf5368dde6cc1bdb0bf55892f263752ecb5e89b"}
Sep 10 23:01:19.205130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"b4a337cf28b04e120830d3e3ad7489ae65a0289788f6893bad39adcd4a13e350"}
Sep 10 23:01:19.205130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204734    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"8facea5cc12d5de0237a6db7c87aed4743bec9b3b66522f76f8f44155acc00ff"}
Sep 10 23:01:19.205130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.204771    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5f858376-5b6b-4da1-b0b7-a19f271f4708","Type":"ContainerStarted","Data":"2b6b4459c355812751460f02665a36486eb0c302712d802ad8f7d872ef35dbf9"}
Sep 10 23:01:19.231080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.231019    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.212016388 podStartE2EDuration="2.230995763s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.979375ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:01:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:01:19.229419928 +0000 UTC m=+201.515836905" watchObservedRunningTime="2026-09-10 23:01:19.230995763 +0000 UTC m=+201.517412711"
Sep 10 23:01:19.917861 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.917829    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"]
Sep 10 23:01:19.921263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.921241    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.923539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923507    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 23:01:19.923539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923519    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 23:01:19.923706 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923548    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 23:01:19.923706 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923584    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-shhhm\""
Sep 10 23:01:19.923706 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923592    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 23:01:19.923873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.923860    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 23:01:19.929016 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.928766    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 23:01:19.932625 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.932603    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"]
Sep 10 23:01:19.967590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967552    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-metrics-client-ca\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967605    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967668    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967705    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967908 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkslq\" (UniqueName: \"kubernetes.io/projected/70fbfabd-574a-489f-90b4-e9c3398f3d8c-kube-api-access-wkslq\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967908 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-federate-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:19.967908 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:19.967843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-serving-certs-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.068873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkslq\" (UniqueName: \"kubernetes.io/projected/70fbfabd-574a-489f-90b4-e9c3398f3d8c-kube-api-access-wkslq\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.068873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-federate-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-serving-certs-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068939    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-metrics-client-ca\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068967    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.068992    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.069022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069126 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.069060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069928 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.069898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-metrics-client-ca\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.069928 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.069898    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-serving-certs-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.070109 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.070077    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.072050 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.072027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.072232 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.072213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-telemeter-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.072232 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.072221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-secret-telemeter-client\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.072363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.072230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/70fbfabd-574a-489f-90b4-e9c3398f3d8c-federate-client-tls\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.078237 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.078208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkslq\" (UniqueName: \"kubernetes.io/projected/70fbfabd-574a-489f-90b4-e9c3398f3d8c-kube-api-access-wkslq\") pod \"telemeter-client-58bfdd7fb6-97j55\" (UID: \"70fbfabd-574a-489f-90b4-e9c3398f3d8c\") " pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.198284 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198250    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:20.198618 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198580    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="prometheus" containerID="cri-o://286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18" gracePeriod=600
Sep 10 23:01:20.198779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198616    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy" containerID="cri-o://c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f" gracePeriod=600
Sep 10 23:01:20.198779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198653    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="config-reloader" containerID="cri-o://9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9" gracePeriod=600
Sep 10 23:01:20.198779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198669    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9" gracePeriod=600
Sep 10 23:01:20.198779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198683    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-web" containerID="cri-o://f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c" gracePeriod=600
Sep 10 23:01:20.198779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.198630    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="thanos-sidecar" containerID="cri-o://3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1" gracePeriod=600
Sep 10 23:01:20.231960 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.231934    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"
Sep 10 23:01:20.436739 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.436711    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:20.513572 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.513532    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-58bfdd7fb6-97j55"]
Sep 10 23:01:20.516340 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:01:20.516275    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70fbfabd_574a_489f_90b4_e9c3398f3d8c.slice/crio-f895f4f5b772cc8a5ba75969a31c157b5418da22adba78f284ea585c8c8961ac WatchSource:0}: Error finding container f895f4f5b772cc8a5ba75969a31c157b5418da22adba78f284ea585c8c8961ac: Status 404 returned error can't find the container with id f895f4f5b772cc8a5ba75969a31c157b5418da22adba78f284ea585c8c8961ac
Sep 10 23:01:20.573246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573205    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lmfb\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573271    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573325    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573406    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573449    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573492    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573525    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573554    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573582    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573607    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573656    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573674    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573684    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573713    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573736    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573758    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573789    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573819    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573858    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.573879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573866    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.574509 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573922    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.574509 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.573951    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out\") pod \"00f9a188-8b65-43b5-a051-56263290f97a\" (UID: \"00f9a188-8b65-43b5-a051-56263290f97a\") "
Sep 10 23:01:20.574509 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.574396    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.574509 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.574448    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.575877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.575851    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.576243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576145    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.576243 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576151    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb" (OuterVolumeSpecName: "kube-api-access-8lmfb") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "kube-api-access-8lmfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:01:20.576686 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576621    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.576823 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576797    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.576987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576891    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.576987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576940    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.576987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576956    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.576987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.576963    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out" (OuterVolumeSpecName: "config-out") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:01:20.577966 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.577510    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.577966 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.577612    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:01:20.577966 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.577917    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config" (OuterVolumeSpecName: "config") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.578682 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.578660    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.578858 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.578829    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:01:20.579154 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.579136    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.579552 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.579529    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:01:20.579648 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.579630    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.588143 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.588111    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config" (OuterVolumeSpecName: "web-config") pod "00f9a188-8b65-43b5-a051-56263290f97a" (UID: "00f9a188-8b65-43b5-a051-56263290f97a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:01:20.675818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675775    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.675818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675808    2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-web-config\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.675818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675817    2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-kube-rbac-proxy\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.675818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675825    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675834    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675845    2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-config\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675853    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675861    2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-tls-assets\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675870    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-metrics-client-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675878    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675886    2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-prometheus-k8s-db\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675894    2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00f9a188-8b65-43b5-a051-56263290f97a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675906    2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/00f9a188-8b65-43b5-a051-56263290f97a-config-out\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675916    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lmfb\" (UniqueName: \"kubernetes.io/projected/00f9a188-8b65-43b5-a051-56263290f97a-kube-api-access-8lmfb\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675924    2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-grpc-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675932    2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675939    2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-secret-metrics-client-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:20.676083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:20.675947    2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/00f9a188-8b65-43b5-a051-56263290f97a-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221123    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9" exitCode=0
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221149    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f" exitCode=0
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221155    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c" exitCode=0
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221161    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1" exitCode=0
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221166    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9" exitCode=0
Sep 10 23:01:21.221165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221173    2565 generic.go:356] "Generic (PLEG): container finished" podID="00f9a188-8b65-43b5-a051-56263290f97a" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18" exitCode=0
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221186    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221222    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221331    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221382    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221423    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221462    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"}
Sep 10 23:01:21.221612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.221501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"00f9a188-8b65-43b5-a051-56263290f97a","Type":"ContainerDied","Data":"3fe23dc72ad0a882f3aa9c7471d85619916fbb2da6680bba1ba2b7e62bed37bf"}
Sep 10 23:01:21.222998 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.222774    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55" event={"ID":"70fbfabd-574a-489f-90b4-e9c3398f3d8c","Type":"ContainerStarted","Data":"f895f4f5b772cc8a5ba75969a31c157b5418da22adba78f284ea585c8c8961ac"}
Sep 10 23:01:21.230756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.230739    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.238771 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.238750    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.248138 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.248105    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:21.248979 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.248956    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.252920 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.252889    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:21.260198 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.260154    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.269990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.269967    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.279266 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279231    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:21.279758 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279737    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="thanos-sidecar"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279761    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="thanos-sidecar"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279778    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="init-config-reloader"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279788    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="init-config-reloader"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279810    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279819    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279829    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-thanos"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279837    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-thanos"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279852    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="config-reloader"
Sep 10 23:01:21.279856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279861    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="config-reloader"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279870    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-web"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279878    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-web"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279888    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="prometheus"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279896    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="prometheus"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279971    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-web"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279983    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="prometheus"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279988    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.279995    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="kube-rbac-proxy-thanos"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.280003    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="config-reloader"
Sep 10 23:01:21.280352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.280007    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00f9a188-8b65-43b5-a051-56263290f97a" containerName="thanos-sidecar"
Sep 10 23:01:21.280808 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.280753    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.285781 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.285756    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.288391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288358    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 23:01:21.288528 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288413    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 23:01:21.288590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288537    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 23:01:21.288642 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288611    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 23:01:21.288690 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288671    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 23:01:21.288758 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288716    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 23:01:21.288823 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.288357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289038    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289077    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wq7xc\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289234    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289352    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 23:01:21.289501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289358    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-am1ffhef871dr\""
Sep 10 23:01:21.289964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.289575    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 23:01:21.291404 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.291390    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 23:01:21.291520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.291498    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.292149 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.292002    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.292149 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.292039    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.292149 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.292064    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.292708 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.292685    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.292786 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.292715    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.292786 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.292737    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.293063 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.293040    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.293147 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293097    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.293147 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293115    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.293507 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.293463    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.293597 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293516    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.293597 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293536    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.293922 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.293898    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.294030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293929    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.294030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.293950    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.294279 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.294230    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.294279 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.294259    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.294458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.294283    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.294854 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:01:21.294828    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.294933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.294895    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.294933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.294917    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.295246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.295218    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.295330 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.295251    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.295743 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.295715    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.295847 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.295744    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.296129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296098    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.296129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296124    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.296251 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296163    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 23:01:21.296451 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296425    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.296507 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296452    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.296765 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296740    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.296831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.296767    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.297180 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297062    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.297180 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297087    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.297941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297473    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.297941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297500    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.298096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297938    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.298096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.297956    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.298841 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.298813    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.298946 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.298862    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.299331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299262    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.299331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299314    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.299607 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299577    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.299607 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299606    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.299911 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299892    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.299911 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.299910    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.300279 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.300254    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.300391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.300281    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.300736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.300713    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:21.300809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.300789    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.300809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.300805    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.301178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301154    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.301239 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301180    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.301518 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301492    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.301595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301519    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.301938 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301823    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.301938 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.301848    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.302170 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302144    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.302269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302171    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.302579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302558    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.302711 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302681    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.302972 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302949    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.303055 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.302972    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.303342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303309    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.303342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303336    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.303669 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303601    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.303669 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303625    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.303983 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303959    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.304051 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.303983    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.304233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.304216    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.304233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.304233    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.304603 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.304572    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.304697 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.304604    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.305040 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.304976    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.305040 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305033    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.305362 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305339    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.305436 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305364    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.305653 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305632    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.305730 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305654    2565 scope.go:117] "RemoveContainer" containerID="2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"
Sep 10 23:01:21.306003 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.305975    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9"} err="failed to get container status \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": rpc error: code = NotFound desc = could not find container \"2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9\": container with ID starting with 2f70f49515fa5db5fa78108f10898255d6faca749b2d8d6835136568582722a9 not found: ID does not exist"
Sep 10 23:01:21.306003 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.306002    2565 scope.go:117] "RemoveContainer" containerID="c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"
Sep 10 23:01:21.306313 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.306262    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f"} err="failed to get container status \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": rpc error: code = NotFound desc = could not find container \"c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f\": container with ID starting with c645834578cf202d24974f1e3771b450039b4fd0d774aa72aa09459ffcd9ae9f not found: ID does not exist"
Sep 10 23:01:21.306313 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.306285    2565 scope.go:117] "RemoveContainer" containerID="f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"
Sep 10 23:01:21.306666 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.306646    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c"} err="failed to get container status \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": rpc error: code = NotFound desc = could not find container \"f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c\": container with ID starting with f3abdd635fa746ab755ccf1d665683ff02e4621c285804fb897a6ec21d67f35c not found: ID does not exist"
Sep 10 23:01:21.306751 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.306667    2565 scope.go:117] "RemoveContainer" containerID="3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"
Sep 10 23:01:21.307282 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307254    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1"} err="failed to get container status \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": rpc error: code = NotFound desc = could not find container \"3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1\": container with ID starting with 3d19cfed1c4bb76f69b0d60918fe42f53909a7be201d2a154b7174b99366d9a1 not found: ID does not exist"
Sep 10 23:01:21.307282 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307280    2565 scope.go:117] "RemoveContainer" containerID="9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"
Sep 10 23:01:21.307600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307580    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9"} err="failed to get container status \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": rpc error: code = NotFound desc = could not find container \"9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9\": container with ID starting with 9581b7f4c19039df435f806e0613893d661d8ebebb084666b15613bdb1b73ec9 not found: ID does not exist"
Sep 10 23:01:21.307686 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307600    2565 scope.go:117] "RemoveContainer" containerID="286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"
Sep 10 23:01:21.307888 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307866    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18"} err="failed to get container status \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": rpc error: code = NotFound desc = could not find container \"286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18\": container with ID starting with 286aa48cd49eab2eb34fbef2e3810ed8f433fa82135fe70921bd77ebffa24a18 not found: ID does not exist"
Sep 10 23:01:21.307968 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.307889    2565 scope.go:117] "RemoveContainer" containerID="fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"
Sep 10 23:01:21.308125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.308097    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74"} err="failed to get container status \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": rpc error: code = NotFound desc = could not find container \"fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74\": container with ID starting with fe153ef12eb6af4a10ec109706f845070d740bf846cc83ca10e7250123f44e74 not found: ID does not exist"
Sep 10 23:01:21.381943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.381851    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.381943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.381888    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.381943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.381916    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.381959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382010    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382046    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382097    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382135    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382187 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382193    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382219    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr9f6\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-kube-api-access-jr9f6\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382240    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382257    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382358    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382387    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382414    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382446    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382473    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.382809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.382508    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483124 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483124 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483269    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483404    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483462    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483498    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483536    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483584    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr9f6\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-kube-api-access-jr9f6\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483614    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483697    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483714    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.483779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.484093 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.484093 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483828    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.484093 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.483867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.484217 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.484159    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.485386 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.484407    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.485386 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.485100    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.485386 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.485111    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.485720    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.486075    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.486916    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.487121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.487564    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.488371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-web-config\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.488449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.488413    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.488905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.489412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a8a2642-a337-4148-81d5-f49696da1b2e-config-out\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.489438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489679 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.489470    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489679 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.489558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1a8a2642-a337-4148-81d5-f49696da1b2e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.489679 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.489602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.490070 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.490048    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1a8a2642-a337-4148-81d5-f49696da1b2e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.499150 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.499118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr9f6\" (UniqueName: \"kubernetes.io/projected/1a8a2642-a337-4148-81d5-f49696da1b2e-kube-api-access-jr9f6\") pod \"prometheus-k8s-0\" (UID: \"1a8a2642-a337-4148-81d5-f49696da1b2e\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.598993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.598938    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:01:21.893403 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:21.893369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 23:01:22.210447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:22.210416    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00f9a188-8b65-43b5-a051-56263290f97a" path="/var/lib/kubelet/pods/00f9a188-8b65-43b5-a051-56263290f97a/volumes"
Sep 10 23:01:22.248014 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:01:22.247987    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8a2642_a337_4148_81d5_f49696da1b2e.slice/crio-90f7f8b1a991b88adfe64682e4c633fa6d7182e2ae990abc9400f43fcb99e83a WatchSource:0}: Error finding container 90f7f8b1a991b88adfe64682e4c633fa6d7182e2ae990abc9400f43fcb99e83a: Status 404 returned error can't find the container with id 90f7f8b1a991b88adfe64682e4c633fa6d7182e2ae990abc9400f43fcb99e83a
Sep 10 23:01:23.235270 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.235228    2565 generic.go:356] "Generic (PLEG): container finished" podID="1a8a2642-a337-4148-81d5-f49696da1b2e" containerID="7780baada06da35c335f33594c96ea9f5f799f20715787ee3c2a5ab13712e48c" exitCode=0
Sep 10 23:01:23.235473 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.235316    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerDied","Data":"7780baada06da35c335f33594c96ea9f5f799f20715787ee3c2a5ab13712e48c"}
Sep 10 23:01:23.235473 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.235438    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"90f7f8b1a991b88adfe64682e4c633fa6d7182e2ae990abc9400f43fcb99e83a"}
Sep 10 23:01:23.237654 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.237629    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55" event={"ID":"70fbfabd-574a-489f-90b4-e9c3398f3d8c","Type":"ContainerStarted","Data":"ecce79fe0da2af1c891eb3e0ce0b2bbe0c62c81b37ed5b2d66a1f298f2f813fd"}
Sep 10 23:01:23.237784 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.237690    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55" event={"ID":"70fbfabd-574a-489f-90b4-e9c3398f3d8c","Type":"ContainerStarted","Data":"426a97ac31f118ec9d01e65214b326b6b78780069866e81653686f35936369f7"}
Sep 10 23:01:23.237784 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.237710    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55" event={"ID":"70fbfabd-574a-489f-90b4-e9c3398f3d8c","Type":"ContainerStarted","Data":"1221773b31fd810139773510a66bbab64f70efbc6f7aed3df3f07c993d7d9ea4"}
Sep 10 23:01:23.284131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:23.284080    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-58bfdd7fb6-97j55" podStartSLOduration=2.501163438 podStartE2EDuration="4.284063467s" totalImagesPullingTime="1.782900029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:01:23.282106084 +0000 UTC m=+205.568523030" watchObservedRunningTime="2026-09-10 23:01:23.284063467 +0000 UTC m=+205.570480414"
Sep 10 23:01:24.246060 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246020    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"1b5f0532492c48bea6a3b729c38f518f12db5753333640e333472f287713bc5c"}
Sep 10 23:01:24.246233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"f74235169fa114d129d069c7092ffe8a031c800f4a4197ccfd54487866205caf"}
Sep 10 23:01:24.246233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246119    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"d5e8014bc5e8fb83d4c770882d2cc13dfbdc413b515c817f3cffb01d7d30d2d0"}
Sep 10 23:01:24.246233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246143    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"5a3aab18ad3d5e5bdaed2d1a8d6935763e12611da994b38748a6da5a3efdd752"}
Sep 10 23:01:24.246233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"a2eddcc30e6abc8b4c762c49ce278620ea3eaa9f1b3e7ed18877fece26ac9e22"}
Sep 10 23:01:24.246233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.246190    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1a8a2642-a337-4148-81d5-f49696da1b2e","Type":"ContainerStarted","Data":"a91b0b9c178fea379faf751f8ab5cfca3346db5967b7b7eb33a35f6fe3b6eda1"}
Sep 10 23:01:24.304822 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:24.304758    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.283729938 podStartE2EDuration="3.304737138s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.0072ms" isStatefulPod=false podCreationTimestamp="2026-09-10 23:01:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:01:24.302169966 +0000 UTC m=+206.588586912" watchObservedRunningTime="2026-09-10 23:01:24.304737138 +0000 UTC m=+206.591154083"
Sep 10 23:01:26.599818 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:01:26.599753    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:02:21.600109 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:02:21.600016    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:02:21.616169 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:02:21.616139    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:02:22.498238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:02:22.498202    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 23:02:58.148498 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:02:58.148462    2565 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 23:03:57.914477 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.914396    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcl5q"]
Sep 10 23:03:57.917530 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.917514    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:57.920359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.920326    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 23:03:57.920488 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.920383    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 23:03:57.921141 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.921121    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8vmhh\""
Sep 10 23:03:57.926222 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:57.926200    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcl5q"]
Sep 10 23:03:58.096904 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.096848    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.097088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.096921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzqt\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-kube-api-access-2dzqt\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.197421 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.197389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzqt\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-kube-api-access-2dzqt\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.197623 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.197457    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.217430 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.217404    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 23:03:58.217959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.217931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.228239 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.228220    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 23:03:58.238363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.238327    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzqt\" (UniqueName: \"kubernetes.io/projected/9edb5d1a-161d-4e83-a8a1-6baa8e202264-kube-api-access-2dzqt\") pod \"cert-manager-webhook-6486ffd997-bcl5q\" (UID: \"9edb5d1a-161d-4e83-a8a1-6baa8e202264\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.535922 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.535831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8vmhh\""
Sep 10 23:03:58.544628 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.544602    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:03:58.823131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.823046    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bcl5q"]
Sep 10 23:03:58.825850 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:03:58.825826    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9edb5d1a_161d_4e83_a8a1_6baa8e202264.slice/crio-442acd9e4bc06108ac640f80a5569d63753a54ca722b6347eec2b6eaf1768f51 WatchSource:0}: Error finding container 442acd9e4bc06108ac640f80a5569d63753a54ca722b6347eec2b6eaf1768f51: Status 404 returned error can't find the container with id 442acd9e4bc06108ac640f80a5569d63753a54ca722b6347eec2b6eaf1768f51
Sep 10 23:03:58.827916 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.827899    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:03:58.861335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:03:58.861306    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q" event={"ID":"9edb5d1a-161d-4e83-a8a1-6baa8e202264","Type":"ContainerStarted","Data":"442acd9e4bc06108ac640f80a5569d63753a54ca722b6347eec2b6eaf1768f51"}
Sep 10 23:04:02.882376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:02.882338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q" event={"ID":"9edb5d1a-161d-4e83-a8a1-6baa8e202264","Type":"ContainerStarted","Data":"4ec50884a529ad01d4d75827a82feb75a6a74a78445b2741c465b44cc2820d79"}
Sep 10 23:04:02.882760 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:02.882438    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:04:02.902165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:02.902113    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q" podStartSLOduration=2.805157668 podStartE2EDuration="5.902091515s" totalImagesPullingTime="3.096933847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:04:02.901683215 +0000 UTC m=+365.188100163" watchObservedRunningTime="2026-09-10 23:04:02.902091515 +0000 UTC m=+365.188508465"
Sep 10 23:04:08.889179 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:08.889147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bcl5q"
Sep 10 23:04:25.842933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.842900    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"]
Sep 10 23:04:25.846012 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.845995    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:25.852269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.852247    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 23:04:25.852426 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.852248    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 23:04:25.852426 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.852340    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 23:04:25.852925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.852629    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jdgl9\""
Sep 10 23:04:25.852925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.852646    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:04:25.854344 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.854326    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 23:04:25.860912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.860892    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"]
Sep 10 23:04:25.930591 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.930559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/995e3521-25d5-45b9-bbc2-3680d3182920-manager-config\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:25.930753 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.930614    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-metrics-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:25.930753 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.930701    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:25.930829 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:25.930783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whp96\" (UniqueName: \"kubernetes.io/projected/995e3521-25d5-45b9-bbc2-3680d3182920-kube-api-access-whp96\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.031778 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.031742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whp96\" (UniqueName: \"kubernetes.io/projected/995e3521-25d5-45b9-bbc2-3680d3182920-kube-api-access-whp96\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.031945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.031786    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/995e3521-25d5-45b9-bbc2-3680d3182920-manager-config\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.031945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.031824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-metrics-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.031945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.031849    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.032617 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.032551    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/995e3521-25d5-45b9-bbc2-3680d3182920-manager-config\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.034543 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.034512    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-metrics-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.034543 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.034515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995e3521-25d5-45b9-bbc2-3680d3182920-cert\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.049341 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.049317    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whp96\" (UniqueName: \"kubernetes.io/projected/995e3521-25d5-45b9-bbc2-3680d3182920-kube-api-access-whp96\") pod \"lws-controller-manager-67777b886f-hgmcj\" (UID: \"995e3521-25d5-45b9-bbc2-3680d3182920\") " pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.155874 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.155783    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:26.438227 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.438203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"]
Sep 10 23:04:26.440839 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:04:26.440811    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995e3521_25d5_45b9_bbc2_3680d3182920.slice/crio-28454d31a094225639db215adcc09cc72ba7052de7923d332c291d51c370e91e WatchSource:0}: Error finding container 28454d31a094225639db215adcc09cc72ba7052de7923d332c291d51c370e91e: Status 404 returned error can't find the container with id 28454d31a094225639db215adcc09cc72ba7052de7923d332c291d51c370e91e
Sep 10 23:04:26.978086 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:26.978051    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj" event={"ID":"995e3521-25d5-45b9-bbc2-3680d3182920","Type":"ContainerStarted","Data":"28454d31a094225639db215adcc09cc72ba7052de7923d332c291d51c370e91e"}
Sep 10 23:04:28.987969 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:28.987881    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj" event={"ID":"995e3521-25d5-45b9-bbc2-3680d3182920","Type":"ContainerStarted","Data":"ea5d2ec6e23e52d573456e59bfd880e78485ea62dd2d019d08901f2c65fc2284"}
Sep 10 23:04:28.988342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:28.988014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:29.014061 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:29.014011    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj" podStartSLOduration=1.717755773 podStartE2EDuration="4.013991491s" totalImagesPullingTime="2.296235718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:04:29.012984629 +0000 UTC m=+391.299401599" watchObservedRunningTime="2026-09-10 23:04:29.013991491 +0000 UTC m=+391.300408475"
Sep 10 23:04:33.643314 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.643270    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"]
Sep 10 23:04:33.646214 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.646191    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.648782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.648758    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 23:04:33.648782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.648773    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 23:04:33.648967 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.648777    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 23:04:33.648967 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.648760    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 23:04:33.649189 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.649174    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dppfd\""
Sep 10 23:04:33.665476 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.665446    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"]
Sep 10 23:04:33.803799 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.803765    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rr86\" (UniqueName: \"kubernetes.io/projected/f57bab69-4315-492d-b1e4-a99246d5999c-kube-api-access-5rr86\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.804000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.803806    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-manifests\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.804000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.803840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.804000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.803917    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-tmp\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.804000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.803968    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905172 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rr86\" (UniqueName: \"kubernetes.io/projected/f57bab69-4315-492d-b1e4-a99246d5999c-kube-api-access-5rr86\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905172 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905129    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-manifests\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905172 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905172 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-tmp\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905475 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.905707 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.905685    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-manifests\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.907741 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.907712    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f57bab69-4315-492d-b1e4-a99246d5999c-tmp\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.907894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.907880    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.908036 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.908013    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f57bab69-4315-492d-b1e4-a99246d5999c-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.923215 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.923193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rr86\" (UniqueName: \"kubernetes.io/projected/f57bab69-4315-492d-b1e4-a99246d5999c-kube-api-access-5rr86\") pod \"opendatahub-operator-controller-manager-dbfbc5c4d-pc92l\" (UID: \"f57bab69-4315-492d-b1e4-a99246d5999c\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:33.957301 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:33.957271    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:34.235600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:34.235537    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"]
Sep 10 23:04:34.238078 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:04:34.238045    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57bab69_4315_492d_b1e4_a99246d5999c.slice/crio-69ee37769d9a4533e0771e28bb5210469e038231db029f6d0f2f5ce4e3ffa586 WatchSource:0}: Error finding container 69ee37769d9a4533e0771e28bb5210469e038231db029f6d0f2f5ce4e3ffa586: Status 404 returned error can't find the container with id 69ee37769d9a4533e0771e28bb5210469e038231db029f6d0f2f5ce4e3ffa586
Sep 10 23:04:35.018752 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:35.018679    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l" event={"ID":"f57bab69-4315-492d-b1e4-a99246d5999c","Type":"ContainerStarted","Data":"69ee37769d9a4533e0771e28bb5210469e038231db029f6d0f2f5ce4e3ffa586"}
Sep 10 23:04:38.034815 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:38.034779    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l" event={"ID":"f57bab69-4315-492d-b1e4-a99246d5999c","Type":"ContainerStarted","Data":"5296428e531a92d735be52755ebbe91422a3e4fd5dc7c6f46e38a30ae534f36e"}
Sep 10 23:04:39.041812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:39.041773    2565 generic.go:356] "Generic (PLEG): container finished" podID="f57bab69-4315-492d-b1e4-a99246d5999c" containerID="5296428e531a92d735be52755ebbe91422a3e4fd5dc7c6f46e38a30ae534f36e" exitCode=0
Sep 10 23:04:39.042326 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:39.041851    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l" event={"ID":"f57bab69-4315-492d-b1e4-a99246d5999c","Type":"ContainerDied","Data":"5296428e531a92d735be52755ebbe91422a3e4fd5dc7c6f46e38a30ae534f36e"}
Sep 10 23:04:39.995251 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:39.995222    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67777b886f-hgmcj"
Sep 10 23:04:40.049083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:40.049052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l" event={"ID":"f57bab69-4315-492d-b1e4-a99246d5999c","Type":"ContainerStarted","Data":"39f1e1966f71f310eb8836456877b13caf8d152024646a8f391475c6a5376ceb"}
Sep 10 23:04:40.049525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:40.049164    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:40.074961 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:40.074913    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l" podStartSLOduration=2.741146562 podStartE2EDuration="7.074897374s" totalImagesPullingTime="3.271854223s" totalInitContainerRuntime="1.061896589s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:04:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:04:40.07446993 +0000 UTC m=+402.360886883" watchObservedRunningTime="2026-09-10 23:04:40.074897374 +0000 UTC m=+402.361314371"
Sep 10 23:04:51.056272 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:51.056237    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfbc5c4d-pc92l"
Sep 10 23:04:59.488464 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.488434    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"]
Sep 10 23:04:59.498303 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.498261    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.501959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.501934    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 23:04:59.502109 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.501962    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n4bs2\""
Sep 10 23:04:59.502109 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.501934    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 23:04:59.502437 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.502420    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 23:04:59.502510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.502420    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 23:04:59.508559 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.508541    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"]
Sep 10 23:04:59.528024 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.527993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6qqr\" (UniqueName: \"kubernetes.io/projected/966b510e-6894-4f2a-ba50-d51034a47d7f-kube-api-access-g6qqr\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.528184 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.528031    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966b510e-6894-4f2a-ba50-d51034a47d7f-tmp\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.528184 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.528137    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.629383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.629350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.629573 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.629396    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6qqr\" (UniqueName: \"kubernetes.io/projected/966b510e-6894-4f2a-ba50-d51034a47d7f-kube-api-access-g6qqr\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.629573 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:04:59.629493    2565 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 23:04:59.629573 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.629517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966b510e-6894-4f2a-ba50-d51034a47d7f-tmp\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.629573 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:04:59.629552    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs podName:966b510e-6894-4f2a-ba50-d51034a47d7f nodeName:}" failed. No retries permitted until 2026-09-10 23:05:00.129536113 +0000 UTC m=+422.415953038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs") pod "kube-auth-proxy-7d9b555765-hzxqd" (UID: "966b510e-6894-4f2a-ba50-d51034a47d7f") : secret "kube-auth-proxy-tls" not found
Sep 10 23:04:59.631812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.631792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/966b510e-6894-4f2a-ba50-d51034a47d7f-tmp\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:04:59.638868 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:04:59.638846    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6qqr\" (UniqueName: \"kubernetes.io/projected/966b510e-6894-4f2a-ba50-d51034a47d7f-kube-api-access-g6qqr\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:05:00.134178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:00.134148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:05:00.136778 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:00.136754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/966b510e-6894-4f2a-ba50-d51034a47d7f-tls-certs\") pod \"kube-auth-proxy-7d9b555765-hzxqd\" (UID: \"966b510e-6894-4f2a-ba50-d51034a47d7f\") " pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:05:00.408310 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:00.408219    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"
Sep 10 23:05:00.689690 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:00.689656    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd"]
Sep 10 23:05:00.692670 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:05:00.692644    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966b510e_6894_4f2a_ba50_d51034a47d7f.slice/crio-2815d0582060474c820aa5a9f734c630b1ab3ff309ca78e5f2cf47c3f15e0b51 WatchSource:0}: Error finding container 2815d0582060474c820aa5a9f734c630b1ab3ff309ca78e5f2cf47c3f15e0b51: Status 404 returned error can't find the container with id 2815d0582060474c820aa5a9f734c630b1ab3ff309ca78e5f2cf47c3f15e0b51
Sep 10 23:05:01.151044 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:01.150957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd" event={"ID":"966b510e-6894-4f2a-ba50-d51034a47d7f","Type":"ContainerStarted","Data":"2815d0582060474c820aa5a9f734c630b1ab3ff309ca78e5f2cf47c3f15e0b51"}
Sep 10 23:05:04.168486 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:04.168448    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd" event={"ID":"966b510e-6894-4f2a-ba50-d51034a47d7f","Type":"ContainerStarted","Data":"bb1c359739728fc8dab219d6dc0f9563fc6411d9387026e295dd033bdab731b8"}
Sep 10 23:05:04.190489 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:05:04.190441    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d9b555765-hzxqd" podStartSLOduration=2.398431426 podStartE2EDuration="5.190423316s" totalImagesPullingTime="2.79199189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:05:04.189228555 +0000 UTC m=+426.475645502" watchObservedRunningTime="2026-09-10 23:05:04.190423316 +0000 UTC m=+426.476840263"
Sep 10 23:06:28.681233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.681197    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-9qvlr"]
Sep 10 23:06:28.684558 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.684541    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:28.690210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.690190    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 23:06:28.690723 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.690687    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-98xx7\""
Sep 10 23:06:28.690837 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.690732    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 23:06:28.707026 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.706997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfs27\" (UniqueName: \"kubernetes.io/projected/394635a7-a40c-411e-83e3-81ac20c50416-kube-api-access-gfs27\") pod \"authorino-operator-76b478bd78-9qvlr\" (UID: \"394635a7-a40c-411e-83e3-81ac20c50416\") " pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:28.718217 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.718189    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-9qvlr"]
Sep 10 23:06:28.808439 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.808401    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfs27\" (UniqueName: \"kubernetes.io/projected/394635a7-a40c-411e-83e3-81ac20c50416-kube-api-access-gfs27\") pod \"authorino-operator-76b478bd78-9qvlr\" (UID: \"394635a7-a40c-411e-83e3-81ac20c50416\") " pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:28.827582 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.827549    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfs27\" (UniqueName: \"kubernetes.io/projected/394635a7-a40c-411e-83e3-81ac20c50416-kube-api-access-gfs27\") pod \"authorino-operator-76b478bd78-9qvlr\" (UID: \"394635a7-a40c-411e-83e3-81ac20c50416\") " pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:28.995178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:28.995088    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:29.285840 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:06:29.285808    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394635a7_a40c_411e_83e3_81ac20c50416.slice/crio-9279da3730dbfa0d75137a14886a8e53a0d131225a19cff3095b97cf006f9a58 WatchSource:0}: Error finding container 9279da3730dbfa0d75137a14886a8e53a0d131225a19cff3095b97cf006f9a58: Status 404 returned error can't find the container with id 9279da3730dbfa0d75137a14886a8e53a0d131225a19cff3095b97cf006f9a58
Sep 10 23:06:29.297403 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:29.297368    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-9qvlr"]
Sep 10 23:06:29.567537 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:29.567455    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr" event={"ID":"394635a7-a40c-411e-83e3-81ac20c50416","Type":"ContainerStarted","Data":"9279da3730dbfa0d75137a14886a8e53a0d131225a19cff3095b97cf006f9a58"}
Sep 10 23:06:32.585769 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:32.585737    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr" event={"ID":"394635a7-a40c-411e-83e3-81ac20c50416","Type":"ContainerStarted","Data":"1f5eaddd821861f7c36d0a76ddcb50f5d48d25930cd82d197a1328e3507b8fa3"}
Sep 10 23:06:32.586153 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:32.585804    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:06:32.623071 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:32.623024    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr" podStartSLOduration=2.143082742 podStartE2EDuration="4.623006406s" totalImagesPullingTime="2.479923664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:06:32.619106413 +0000 UTC m=+514.905523370" watchObservedRunningTime="2026-09-10 23:06:32.623006406 +0000 UTC m=+514.909423352"
Sep 10 23:06:43.591735 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:06:43.591697    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-9qvlr"
Sep 10 23:07:17.590717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.590619    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:07:17.594258 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.594234    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.596714 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.596683    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 23:07:17.596893 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.596877    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vpt4b\""
Sep 10 23:07:17.614934 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.614908    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:07:17.649669 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.649636    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:07:17.735201 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.735166    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs2rw\" (UniqueName: \"kubernetes.io/projected/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-kube-api-access-xs2rw\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.735201 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.735199    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-config-file\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.836204 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.836167    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs2rw\" (UniqueName: \"kubernetes.io/projected/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-kube-api-access-xs2rw\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.836204 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.836208    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-config-file\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.836848 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.836830    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-config-file\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.846477 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.846412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs2rw\" (UniqueName: \"kubernetes.io/projected/8a0be99c-0a8d-4dd1-a720-4f57498d3c75-kube-api-access-xs2rw\") pod \"limitador-limitador-84994f7ffc-595v2\" (UID: \"8a0be99c-0a8d-4dd1-a720-4f57498d3c75\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:17.904016 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:17.903970    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:18.187556 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.187533    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:07:18.190273 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:07:18.190244    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0be99c_0a8d_4dd1_a720_4f57498d3c75.slice/crio-d653026311a612f69b35cbe7ff1ee59dd188100c4a034c949ff413e7f38fe19e WatchSource:0}: Error finding container d653026311a612f69b35cbe7ff1ee59dd188100c4a034c949ff413e7f38fe19e: Status 404 returned error can't find the container with id d653026311a612f69b35cbe7ff1ee59dd188100c4a034c949ff413e7f38fe19e
Sep 10 23:07:18.422145 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.422110    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:18.426785 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.426767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:18.429241 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.429223    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zzgsr\""
Sep 10 23:07:18.439073 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.439020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:18.542204 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.542176    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9r4z\" (UniqueName: \"kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z\") pod \"authorino-59c974bd48-fg8bl\" (UID: \"c10442a5-90ab-434e-a555-8e172c4ef6c7\") " pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:18.642844 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.642810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9r4z\" (UniqueName: \"kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z\") pod \"authorino-59c974bd48-fg8bl\" (UID: \"c10442a5-90ab-434e-a555-8e172c4ef6c7\") " pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:18.658933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.658909    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9r4z\" (UniqueName: \"kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z\") pod \"authorino-59c974bd48-fg8bl\" (UID: \"c10442a5-90ab-434e-a555-8e172c4ef6c7\") " pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:18.735988 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.735901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:18.804970 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:18.804859    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2" event={"ID":"8a0be99c-0a8d-4dd1-a720-4f57498d3c75","Type":"ContainerStarted","Data":"d653026311a612f69b35cbe7ff1ee59dd188100c4a034c949ff413e7f38fe19e"}
Sep 10 23:07:19.027116 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:19.027085    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:19.031267 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:07:19.031232    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc10442a5_90ab_434e_a555_8e172c4ef6c7.slice/crio-b8e56e174f17aeb6480b520867d8d7f4d0355afe2d088bf200809acf7f8ce36c WatchSource:0}: Error finding container b8e56e174f17aeb6480b520867d8d7f4d0355afe2d088bf200809acf7f8ce36c: Status 404 returned error can't find the container with id b8e56e174f17aeb6480b520867d8d7f4d0355afe2d088bf200809acf7f8ce36c
Sep 10 23:07:19.812555 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:19.812518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2" event={"ID":"8a0be99c-0a8d-4dd1-a720-4f57498d3c75","Type":"ContainerStarted","Data":"f3cbf3b65843283888f5343622a597d8f6dcb5791f7953e0f072862f73514cf5"}
Sep 10 23:07:19.813017 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:19.812609    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:19.813903 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:19.813875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fg8bl" event={"ID":"c10442a5-90ab-434e-a555-8e172c4ef6c7","Type":"ContainerStarted","Data":"b8e56e174f17aeb6480b520867d8d7f4d0355afe2d088bf200809acf7f8ce36c"}
Sep 10 23:07:19.834154 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:19.834094    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2" podStartSLOduration=1.940415751 podStartE2EDuration="2.834071532s" totalImagesPullingTime="893.655781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:07:19.831713786 +0000 UTC m=+562.118130733" watchObservedRunningTime="2026-09-10 23:07:19.834071532 +0000 UTC m=+562.120488482"
Sep 10 23:07:20.822838 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:20.822794    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fg8bl" event={"ID":"c10442a5-90ab-434e-a555-8e172c4ef6c7","Type":"ContainerStarted","Data":"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"}
Sep 10 23:07:20.844102 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:20.844035    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-fg8bl" podStartSLOduration=1.225041955 podStartE2EDuration="2.844012323s" totalImagesPullingTime="1.618970368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:07:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:07:20.841509155 +0000 UTC m=+563.127926102" watchObservedRunningTime="2026-09-10 23:07:20.844012323 +0000 UTC m=+563.130429273"
Sep 10 23:07:30.823821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:30.823791    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-595v2"
Sep 10 23:07:48.577403 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.577360    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:48.578063 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.577557    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-fg8bl" podUID="c10442a5-90ab-434e-a555-8e172c4ef6c7" containerName="authorino" containerID="cri-o://568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f" gracePeriod=30
Sep 10 23:07:48.821115 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.821091    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:48.909520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.909439    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9r4z\" (UniqueName: \"kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z\") pod \"c10442a5-90ab-434e-a555-8e172c4ef6c7\" (UID: \"c10442a5-90ab-434e-a555-8e172c4ef6c7\") "
Sep 10 23:07:48.911742 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.911710    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z" (OuterVolumeSpecName: "kube-api-access-g9r4z") pod "c10442a5-90ab-434e-a555-8e172c4ef6c7" (UID: "c10442a5-90ab-434e-a555-8e172c4ef6c7"). InnerVolumeSpecName "kube-api-access-g9r4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:07:48.951080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.951046    2565 generic.go:356] "Generic (PLEG): container finished" podID="c10442a5-90ab-434e-a555-8e172c4ef6c7" containerID="568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f" exitCode=0
Sep 10 23:07:48.951210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.951095    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-fg8bl"
Sep 10 23:07:48.951210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.951091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fg8bl" event={"ID":"c10442a5-90ab-434e-a555-8e172c4ef6c7","Type":"ContainerDied","Data":"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"}
Sep 10 23:07:48.951210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.951135    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-fg8bl" event={"ID":"c10442a5-90ab-434e-a555-8e172c4ef6c7","Type":"ContainerDied","Data":"b8e56e174f17aeb6480b520867d8d7f4d0355afe2d088bf200809acf7f8ce36c"}
Sep 10 23:07:48.951210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.951149    2565 scope.go:117] "RemoveContainer" containerID="568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"
Sep 10 23:07:48.960251 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.960235    2565 scope.go:117] "RemoveContainer" containerID="568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"
Sep 10 23:07:48.960567 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:07:48.960548    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f\": container with ID starting with 568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f not found: ID does not exist" containerID="568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"
Sep 10 23:07:48.960644 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.960577    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f"} err="failed to get container status \"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f\": rpc error: code = NotFound desc = could not find container \"568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f\": container with ID starting with 568cbe6c4080b0163274eda9881f1273c4c4d5c3c8324ca38518d5645150987f not found: ID does not exist"
Sep 10 23:07:48.981225 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.976125    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:48.983411 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:48.983390    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-fg8bl"]
Sep 10 23:07:49.010282 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:49.010245    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9r4z\" (UniqueName: \"kubernetes.io/projected/c10442a5-90ab-434e-a555-8e172c4ef6c7-kube-api-access-g9r4z\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:07:50.210001 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:07:50.209970    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c10442a5-90ab-434e-a555-8e172c4ef6c7" path="/var/lib/kubelet/pods/c10442a5-90ab-434e-a555-8e172c4ef6c7/volumes"
Sep 10 23:08:30.295694 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:30.295611    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:08:51.678129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.678092    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"]
Sep 10 23:08:51.678565 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.678518    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c10442a5-90ab-434e-a555-8e172c4ef6c7" containerName="authorino"
Sep 10 23:08:51.678565 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.678534    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c10442a5-90ab-434e-a555-8e172c4ef6c7" containerName="authorino"
Sep 10 23:08:51.678635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.678610    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c10442a5-90ab-434e-a555-8e172c4ef6c7" containerName="authorino"
Sep 10 23:08:51.681657 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.681641    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.683925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.683898    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 23:08:51.684073 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.683957    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-85z7r\""
Sep 10 23:08:51.684073 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.684069    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 23:08:51.684622 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.684604    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 23:08:51.692849 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.692827    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"]
Sep 10 23:08:51.748310 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.748470 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.748470 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748388    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxxtl\" (UniqueName: \"kubernetes.io/projected/ff0cc5d6-d73c-4518-a1f0-f73873d691de-kube-api-access-bxxtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.748470 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.748470 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748449    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.748601 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.748517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849664 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849626    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849684    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxxtl\" (UniqueName: \"kubernetes.io/projected/ff0cc5d6-d73c-4518-a1f0-f73873d691de-kube-api-access-bxxtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849761    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.849839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.849811    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.850110 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.850086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.850166 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.850144    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.850166 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.850156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.852349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.852323    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff0cc5d6-d73c-4518-a1f0-f73873d691de-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.852499 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.852481    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff0cc5d6-d73c-4518-a1f0-f73873d691de-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.860406 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.860385    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxxtl\" (UniqueName: \"kubernetes.io/projected/ff0cc5d6-d73c-4518-a1f0-f73873d691de-kube-api-access-bxxtl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc\" (UID: \"ff0cc5d6-d73c-4518-a1f0-f73873d691de\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:51.992431 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:51.992355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:08:52.179285 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:52.179250    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:08:52.271674 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:52.271651    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"]
Sep 10 23:08:52.273269 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:08:52.273241    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0cc5d6_d73c_4518_a1f0_f73873d691de.slice/crio-d9f779f140eab4659a1b395230e0c5cdd7ee5d6a238966bc1736405f83b9072e WatchSource:0}: Error finding container d9f779f140eab4659a1b395230e0c5cdd7ee5d6a238966bc1736405f83b9072e: Status 404 returned error can't find the container with id d9f779f140eab4659a1b395230e0c5cdd7ee5d6a238966bc1736405f83b9072e
Sep 10 23:08:53.242529 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:53.242492    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc" event={"ID":"ff0cc5d6-d73c-4518-a1f0-f73873d691de","Type":"ContainerStarted","Data":"d9f779f140eab4659a1b395230e0c5cdd7ee5d6a238966bc1736405f83b9072e"}
Sep 10 23:08:54.248260 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:54.248230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc" event={"ID":"ff0cc5d6-d73c-4518-a1f0-f73873d691de","Type":"ContainerStarted","Data":"9adf0101eb7b354ffa05aed04cdfca042bd2add8a43a10b293cf593e32dc106f"}
Sep 10 23:08:54.274813 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:54.274766    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc" podStartSLOduration=2.12042861 podStartE2EDuration="3.274751323s" totalImagesPullingTime="1.154322713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:08:54.274364581 +0000 UTC m=+656.560781529" watchObservedRunningTime="2026-09-10 23:08:54.274751323 +0000 UTC m=+656.561168267"
Sep 10 23:08:58.697562 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:08:58.697529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:01.992816 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:01.992780    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:09:01.993241 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:01.993116    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:09:02.005070 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:02.005042    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:09:02.296834 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:02.296751    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc"
Sep 10 23:09:13.190746 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:13.190710    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:16.368047 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.368009    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"]
Sep 10 23:09:16.374434 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.374402    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.377840 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.377812    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 23:09:16.386691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.386662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"]
Sep 10 23:09:16.462683 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddpjh\" (UniqueName: \"kubernetes.io/projected/b27ac81e-1254-4343-9abe-a81d687580af-kube-api-access-ddpjh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.462683 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462692    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.462918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27ac81e-1254-4343-9abe-a81d687580af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.462918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.462918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.462918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.462855    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.563796 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddpjh\" (UniqueName: \"kubernetes.io/projected/b27ac81e-1254-4343-9abe-a81d687580af-kube-api-access-ddpjh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.563957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563815    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.563957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563857    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27ac81e-1254-4343-9abe-a81d687580af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.563957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563901    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.563957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.564127 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.563966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.564347 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.564321    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.564443 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.564325    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.564505 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.564458    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.566349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.566331    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b27ac81e-1254-4343-9abe-a81d687580af-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.566517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.566500    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b27ac81e-1254-4343-9abe-a81d687580af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.575246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.575222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddpjh\" (UniqueName: \"kubernetes.io/projected/b27ac81e-1254-4343-9abe-a81d687580af-kube-api-access-ddpjh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-z4zwt\" (UID: \"b27ac81e-1254-4343-9abe-a81d687580af\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.685385 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.685358    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:16.958897 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.958869    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"]
Sep 10 23:09:16.960920 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:09:16.960893    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27ac81e_1254_4343_9abe_a81d687580af.slice/crio-638a37fc751c59b4efc1cd752152634379940b6eb29eca2300dd2bcd315efa5c WatchSource:0}: Error finding container 638a37fc751c59b4efc1cd752152634379940b6eb29eca2300dd2bcd315efa5c: Status 404 returned error can't find the container with id 638a37fc751c59b4efc1cd752152634379940b6eb29eca2300dd2bcd315efa5c
Sep 10 23:09:16.962235 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:16.962217    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:09:17.362812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:17.362715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt" event={"ID":"b27ac81e-1254-4343-9abe-a81d687580af","Type":"ContainerStarted","Data":"d263392c3e8bd798ae7226585d7d2a7e9ccba84a559862165948adb859631756"}
Sep 10 23:09:17.362812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:17.362782    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt" event={"ID":"b27ac81e-1254-4343-9abe-a81d687580af","Type":"ContainerStarted","Data":"638a37fc751c59b4efc1cd752152634379940b6eb29eca2300dd2bcd315efa5c"}
Sep 10 23:09:17.386938 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:17.386852    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt" podStartSLOduration=1.211451447 podStartE2EDuration="1.386830934s" totalImagesPullingTime="175.379487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:09:17.385214636 +0000 UTC m=+679.671631582" watchObservedRunningTime="2026-09-10 23:09:17.386830934 +0000 UTC m=+679.673247884"
Sep 10 23:09:17.580266 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:17.580223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:26.382448 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:26.382409    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:26.685723 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:26.685683    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:26.685900 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:26.685743    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:26.698212 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:26.698183    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:27.421565 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:27.421537    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-z4zwt"
Sep 10 23:09:30.773602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.773564    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"]
Sep 10 23:09:30.777231 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.777211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.779556 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.779534    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 23:09:30.788258 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.788234    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"]
Sep 10 23:09:30.896647 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896612    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcc5v\" (UniqueName: \"kubernetes.io/projected/294e8555-fe18-4f99-a48e-2228301d8875-kube-api-access-wcc5v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.896855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e8555-fe18-4f99-a48e-2228301d8875-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.896855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896698    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.896855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896739    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.896855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.896855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.896787    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.997994 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.997957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcc5v\" (UniqueName: \"kubernetes.io/projected/294e8555-fe18-4f99-a48e-2228301d8875-kube-api-access-wcc5v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998104 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998007    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e8555-fe18-4f99-a48e-2228301d8875-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998104 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998104 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998104 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998132    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998561 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998537    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998647 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:30.998688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:30.998647    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:31.000602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.000575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/294e8555-fe18-4f99-a48e-2228301d8875-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:31.000795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.000780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/294e8555-fe18-4f99-a48e-2228301d8875-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:31.007602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.007576    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcc5v\" (UniqueName: \"kubernetes.io/projected/294e8555-fe18-4f99-a48e-2228301d8875-kube-api-access-wcc5v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-clkgr\" (UID: \"294e8555-fe18-4f99-a48e-2228301d8875\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:31.089041 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.088946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:31.368335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.368220    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"]
Sep 10 23:09:31.371146 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:09:31.371118    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294e8555_fe18_4f99_a48e_2228301d8875.slice/crio-679cba2eb02af81414728f2938f5775cb26720dc839e2cd46f1420154ee6441c WatchSource:0}: Error finding container 679cba2eb02af81414728f2938f5775cb26720dc839e2cd46f1420154ee6441c: Status 404 returned error can't find the container with id 679cba2eb02af81414728f2938f5775cb26720dc839e2cd46f1420154ee6441c
Sep 10 23:09:31.431561 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:31.431526    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr" event={"ID":"294e8555-fe18-4f99-a48e-2228301d8875","Type":"ContainerStarted","Data":"679cba2eb02af81414728f2938f5775cb26720dc839e2cd46f1420154ee6441c"}
Sep 10 23:09:32.443997 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:32.443963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr" event={"ID":"294e8555-fe18-4f99-a48e-2228301d8875","Type":"ContainerStarted","Data":"30741503e376ea3807cee0effa732ffb80e500d5648ac9f7f3541c093d9702b0"}
Sep 10 23:09:32.465737 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:32.465687    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr" podStartSLOduration=2.284853063 podStartE2EDuration="2.46566871s" totalImagesPullingTime="180.815647ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:09:32.46424838 +0000 UTC m=+694.750665326" watchObservedRunningTime="2026-09-10 23:09:32.46566871 +0000 UTC m=+694.752085658"
Sep 10 23:09:32.864426 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:32.864353    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:39.765473 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.765436    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"]
Sep 10 23:09:39.769233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.769211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.771526 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.771505    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 23:09:39.787312 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.787252    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"]
Sep 10 23:09:39.877937 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.877903    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.878095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.877948    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a9812e-85d6-495f-99b5-630e413eb6fa-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.878095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.877966    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.878095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.878076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.878223 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.878106    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.878223 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.878133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrrsm\" (UniqueName: \"kubernetes.io/projected/70a9812e-85d6-495f-99b5-630e413eb6fa-kube-api-access-wrrsm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978639 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978609    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978654    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a9812e-85d6-495f-99b5-630e413eb6fa-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.978795 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.978753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrrsm\" (UniqueName: \"kubernetes.io/projected/70a9812e-85d6-495f-99b5-630e413eb6fa-kube-api-access-wrrsm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.979163 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.979135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.979163 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.979150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.979336 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.979188    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.981039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.981018    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a9812e-85d6-495f-99b5-630e413eb6fa-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.981368 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.981351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a9812e-85d6-495f-99b5-630e413eb6fa-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:39.988375 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:39.988351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrrsm\" (UniqueName: \"kubernetes.io/projected/70a9812e-85d6-495f-99b5-630e413eb6fa-kube-api-access-wrrsm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9fllh\" (UID: \"70a9812e-85d6-495f-99b5-630e413eb6fa\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:40.079997 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:40.079904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:40.360981 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:40.360898    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"]
Sep 10 23:09:40.364196 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:09:40.364171    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70a9812e_85d6_495f_99b5_630e413eb6fa.slice/crio-3762a74d3729b9c399ce093ce65abbc22ae0afc4a080aee650e139ab91b1df35 WatchSource:0}: Error finding container 3762a74d3729b9c399ce093ce65abbc22ae0afc4a080aee650e139ab91b1df35: Status 404 returned error can't find the container with id 3762a74d3729b9c399ce093ce65abbc22ae0afc4a080aee650e139ab91b1df35
Sep 10 23:09:40.481474 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:40.481440    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh" event={"ID":"70a9812e-85d6-495f-99b5-630e413eb6fa","Type":"ContainerStarted","Data":"3762a74d3729b9c399ce093ce65abbc22ae0afc4a080aee650e139ab91b1df35"}
Sep 10 23:09:41.089392 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.089351    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:41.089769 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.089426    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:41.101912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.101880    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:41.487572 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.487536    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh" event={"ID":"70a9812e-85d6-495f-99b5-630e413eb6fa","Type":"ContainerStarted","Data":"14b6de69ba4948e6bbe31899963f685b52877833f2cc47e1e0718e9ee3795e38"}
Sep 10 23:09:41.498870 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.498838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-clkgr"
Sep 10 23:09:41.509139 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:41.509090    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh" podStartSLOduration=2.328683552 podStartE2EDuration="2.509075853s" totalImagesPullingTime="180.392301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:09:41.506643888 +0000 UTC m=+703.793060835" watchObservedRunningTime="2026-09-10 23:09:41.509075853 +0000 UTC m=+703.795492799"
Sep 10 23:09:42.169100 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:42.169067    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:09:50.080240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:50.080150    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:50.080240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:50.080226    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:50.092499 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:50.092473    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:09:50.544579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:09:50.544536    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9fllh"
Sep 10 23:11:37.966095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:37.966061    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c979f8d77-crf2j"]
Sep 10 23:11:37.969567 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:37.969548    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:37.972462 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:37.972434    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zzgsr\""
Sep 10 23:11:37.972595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:37.972434    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 23:11:37.987221 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:37.987191    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c979f8d77-crf2j"]
Sep 10 23:11:38.060175 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.060144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jbp4\" (UniqueName: \"kubernetes.io/projected/5fc1ca76-9474-4173-a533-cfb06fb946c8-kube-api-access-5jbp4\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.060367 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.060201    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5fc1ca76-9474-4173-a533-cfb06fb946c8-tls-cert\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.161480 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.161446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5fc1ca76-9474-4173-a533-cfb06fb946c8-tls-cert\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.161657 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.161523    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jbp4\" (UniqueName: \"kubernetes.io/projected/5fc1ca76-9474-4173-a533-cfb06fb946c8-kube-api-access-5jbp4\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.164142 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.164114    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5fc1ca76-9474-4173-a533-cfb06fb946c8-tls-cert\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.172495 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.172465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jbp4\" (UniqueName: \"kubernetes.io/projected/5fc1ca76-9474-4173-a533-cfb06fb946c8-kube-api-access-5jbp4\") pod \"authorino-c979f8d77-crf2j\" (UID: \"5fc1ca76-9474-4173-a533-cfb06fb946c8\") " pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.278476 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.278392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c979f8d77-crf2j"
Sep 10 23:11:38.551683 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:38.551656    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c979f8d77-crf2j"]
Sep 10 23:11:38.554156 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:11:38.554128    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fc1ca76_9474_4173_a533_cfb06fb946c8.slice/crio-a847c9f06bf37334f582fd68c6b6603bb0434f252a6829b1474277f185d81cf2 WatchSource:0}: Error finding container a847c9f06bf37334f582fd68c6b6603bb0434f252a6829b1474277f185d81cf2: Status 404 returned error can't find the container with id a847c9f06bf37334f582fd68c6b6603bb0434f252a6829b1474277f185d81cf2
Sep 10 23:11:39.052760 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:39.052722    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c979f8d77-crf2j" event={"ID":"5fc1ca76-9474-4173-a533-cfb06fb946c8","Type":"ContainerStarted","Data":"a847c9f06bf37334f582fd68c6b6603bb0434f252a6829b1474277f185d81cf2"}
Sep 10 23:11:40.060103 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:40.060063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c979f8d77-crf2j" event={"ID":"5fc1ca76-9474-4173-a533-cfb06fb946c8","Type":"ContainerStarted","Data":"6ce3ed0196b8569e74dbdafc9211be9804eb26d386761bf26c7ad2078901a6d2"}
Sep 10 23:11:40.087143 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:11:40.087091    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c979f8d77-crf2j" podStartSLOduration=2.248607075 podStartE2EDuration="3.087074665s" totalImagesPullingTime="838.46759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:11:40.083203054 +0000 UTC m=+822.369620002" watchObservedRunningTime="2026-09-10 23:11:40.087074665 +0000 UTC m=+822.373491612"
Sep 10 23:12:01.478133 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:01.478095    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:12:16.082538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:16.082494    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:12:35.504821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.504789    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"]
Sep 10 23:12:35.508285 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.508268    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:35.510782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.510760    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:12:35.511050 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.511029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:12:35.518784 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.518749    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"]
Sep 10 23:12:35.622926 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.622891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs\" (UID: \"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:35.686490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.686454    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:12:35.724271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.724231    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs\" (UID: \"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:35.724899 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.724876    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs\" (UID: \"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:35.819941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:35.819852    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:36.108802 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:36.108723    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"]
Sep 10 23:12:36.112412 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:12:36.112384    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe7bcb7_f7d1_4cac_a2d8_6712ff3ab593.slice/crio-8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4 WatchSource:0}: Error finding container 8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4: Status 404 returned error can't find the container with id 8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4
Sep 10 23:12:36.336671 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:36.336637    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs" event={"ID":"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593","Type":"ContainerStarted","Data":"8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4"}
Sep 10 23:12:39.356989 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:39.356959    2565 generic.go:356] "Generic (PLEG): container finished" podID="bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" containerID="5ea6f9bbd58137380fb7e1a83fb54f24d934916c56259723533c7b7acef20e80" exitCode=0
Sep 10 23:12:39.357358 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:39.357049    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs" event={"ID":"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593","Type":"ContainerDied","Data":"5ea6f9bbd58137380fb7e1a83fb54f24d934916c56259723533c7b7acef20e80"}
Sep 10 23:12:40.493049 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:40.493023    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:40.673383 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:40.673362    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca\") pod \"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593\" (UID: \"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593\") "
Sep 10 23:12:40.673690 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:40.673667    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" (UID: "bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:12:40.773964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:40.773933    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:12:41.367651 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:41.367626    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"
Sep 10 23:12:41.367827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:41.367624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs" event={"ID":"bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593","Type":"ContainerDied","Data":"8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4"}
Sep 10 23:12:41.367827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:41.367749    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d05102b98880fa85ce63f8a533e8e810f36b3cf3dcd6e6d42b9849971508ba4"
Sep 10 23:12:50.666238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:50.666149    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:12:52.507273 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:52.507240    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"]
Sep 10 23:12:52.509831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:52.509806    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e7db0e72-rpccs"]
Sep 10 23:12:54.209651 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:12:54.209618    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" path="/var/lib/kubelet/pods/bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593/volumes"
Sep 10 23:13:09.895571 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:09.895537    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:13:10.467517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.467473    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:13:10.467887 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.467873    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" containerName="revoke-keys"
Sep 10 23:13:10.467932 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.467889    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" containerName="revoke-keys"
Sep 10 23:13:10.467974 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.467965    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe7bcb7-f7d1-4cac-a2d8-6712ff3ab593" containerName="revoke-keys"
Sep 10 23:13:10.473485 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.473463    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.476545 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.476512    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-13cbbb71-kserve-self-signed-certs\""
Sep 10 23:13:10.487642 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.487494    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:13:10.550664 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550633    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.550820 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550678    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.550820 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550769    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.550820 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.550820 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550816    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxqtg\" (UniqueName: \"kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.551026 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.550892    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652206 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652173    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652206 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652205    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxqtg\" (UniqueName: \"kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652250    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652286    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652735 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652710    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652808 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652748    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.652869 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.652822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.654763 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.654742    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.655014 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.654995    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.667391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.667367    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxqtg\" (UniqueName: \"kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg\") pod \"e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") " pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:10.782222 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:10.782133    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:11.060809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:11.060734    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:13:11.064128 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:13:11.064094    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89396197_cf5a_446e_a52d_70767fb65176.slice/crio-7ca460baa9de43b4c77e88b9c84f4d8b6b4a843e68126f3bd2010153a2d691bf WatchSource:0}: Error finding container 7ca460baa9de43b4c77e88b9c84f4d8b6b4a843e68126f3bd2010153a2d691bf: Status 404 returned error can't find the container with id 7ca460baa9de43b4c77e88b9c84f4d8b6b4a843e68126f3bd2010153a2d691bf
Sep 10 23:13:11.518546 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:11.518509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" event={"ID":"89396197-cf5a-446e-a52d-70767fb65176","Type":"ContainerStarted","Data":"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"}
Sep 10 23:13:11.518546 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:11.518551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" event={"ID":"89396197-cf5a-446e-a52d-70767fb65176","Type":"ContainerStarted","Data":"7ca460baa9de43b4c77e88b9c84f4d8b6b4a843e68126f3bd2010153a2d691bf"}
Sep 10 23:13:11.544856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:11.544804    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" podStartSLOduration=1.5447869889999999 podStartE2EDuration="1.544786989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:13:11.542653299 +0000 UTC m=+913.829070257" watchObservedRunningTime="2026-09-10 23:13:11.544786989 +0000 UTC m=+913.831203936"
Sep 10 23:13:20.782371 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:20.782330    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:20.782371 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:20.782375    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:20.794828 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:20.794805    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:21.418437 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.418406    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"]
Sep 10 23:13:21.421793 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.421776    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:21.425011 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.424988    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:13:21.425135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.424995    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:13:21.435040 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.435013    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"]
Sep 10 23:13:21.558061 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.558030    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk\" (UID: \"5be553ec-be70-4613-ac02-c0183eac39bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:21.581846 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.581814    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:13:21.659013 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.658977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk\" (UID: \"5be553ec-be70-4613-ac02-c0183eac39bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:21.659591 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.659574    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk\" (UID: \"5be553ec-be70-4613-ac02-c0183eac39bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:21.732885 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:21.732856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:22.014772 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:22.014742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"]
Sep 10 23:13:22.017674 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:13:22.017647    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be553ec_be70_4613_ac02_c0183eac39bc.slice/crio-71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5 WatchSource:0}: Error finding container 71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5: Status 404 returned error can't find the container with id 71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5
Sep 10 23:13:22.576480 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:22.576390    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk" event={"ID":"5be553ec-be70-4613-ac02-c0183eac39bc","Type":"ContainerStarted","Data":"b7a03a11ab31e693f6f5f2ec7659f4f474d28fa78997c6b877eb8dc31a193f09"}
Sep 10 23:13:22.576480 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:22.576447    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk" event={"ID":"5be553ec-be70-4613-ac02-c0183eac39bc","Type":"ContainerStarted","Data":"71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5"}
Sep 10 23:13:22.592131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:22.592065    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk" podStartSLOduration=1.592048717 podStartE2EDuration="1.592048717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:13:22.591665198 +0000 UTC m=+924.878082145" watchObservedRunningTime="2026-09-10 23:13:22.592048717 +0000 UTC m=+924.878465663"
Sep 10 23:13:23.583082 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:23.583051    2565 generic.go:356] "Generic (PLEG): container finished" podID="5be553ec-be70-4613-ac02-c0183eac39bc" containerID="b7a03a11ab31e693f6f5f2ec7659f4f474d28fa78997c6b877eb8dc31a193f09" exitCode=0
Sep 10 23:13:23.583594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:23.583101    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk" event={"ID":"5be553ec-be70-4613-ac02-c0183eac39bc","Type":"ContainerDied","Data":"b7a03a11ab31e693f6f5f2ec7659f4f474d28fa78997c6b877eb8dc31a193f09"}
Sep 10 23:13:24.724794 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:24.724772    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:24.786345 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:24.786283    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca\") pod \"5be553ec-be70-4613-ac02-c0183eac39bc\" (UID: \"5be553ec-be70-4613-ac02-c0183eac39bc\") "
Sep 10 23:13:24.786667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:24.786644    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5be553ec-be70-4613-ac02-c0183eac39bc" (UID: "5be553ec-be70-4613-ac02-c0183eac39bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:13:24.887443 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:24.887361    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be553ec-be70-4613-ac02-c0183eac39bc-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:13:25.593222 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:25.593192    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"
Sep 10 23:13:25.593222 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:25.593209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk" event={"ID":"5be553ec-be70-4613-ac02-c0183eac39bc","Type":"ContainerDied","Data":"71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5"}
Sep 10 23:13:25.593502 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:25.593247    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71868b10abd24b55c7b1589cf6452e0c1ef8dedaf2b0442706d43cb3bef185c5"
Sep 10 23:13:27.362477 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:27.362433    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:13:28.365131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:28.365098    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"]
Sep 10 23:13:28.368271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:28.368246    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-4a0c9f43-mpfbk"]
Sep 10 23:13:30.210486 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:30.210454    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be553ec-be70-4613-ac02-c0183eac39bc" path="/var/lib/kubelet/pods/5be553ec-be70-4613-ac02-c0183eac39bc/volumes"
Sep 10 23:13:42.862698 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:42.862662    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:13:57.414380 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.414345    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"]
Sep 10 23:13:57.414881 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.414750    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be553ec-be70-4613-ac02-c0183eac39bc" containerName="revoke-keys"
Sep 10 23:13:57.414881 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.414762    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be553ec-be70-4613-ac02-c0183eac39bc" containerName="revoke-keys"
Sep 10 23:13:57.414881 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.414842    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be553ec-be70-4613-ac02-c0183eac39bc" containerName="revoke-keys"
Sep 10 23:13:57.417941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.417923    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:57.420205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.420180    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:13:57.420371 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.420180    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:13:57.426778 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.426748    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"]
Sep 10 23:13:57.589943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.589914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-00575c78-zwzzd\" (UID: \"08cf40f5-5731-4996-b824-bb2975c3eb4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:57.690730 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.690698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-00575c78-zwzzd\" (UID: \"08cf40f5-5731-4996-b824-bb2975c3eb4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:57.691246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.691227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-00575c78-zwzzd\" (UID: \"08cf40f5-5731-4996-b824-bb2975c3eb4f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:57.730298 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:57.730269    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:58.001858 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:58.001833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"]
Sep 10 23:13:58.003637 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:13:58.003609    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cf40f5_5731_4996_b824_bb2975c3eb4f.slice/crio-3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1 WatchSource:0}: Error finding container 3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1: Status 404 returned error can't find the container with id 3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1
Sep 10 23:13:58.763426 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:58.763394    2565 generic.go:356] "Generic (PLEG): container finished" podID="08cf40f5-5731-4996-b824-bb2975c3eb4f" containerID="5fda273dad65a5ca1f904dd87005205b74572be97076dbc6ae4c9a621cf4baad" exitCode=0
Sep 10 23:13:58.763858 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:58.763477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd" event={"ID":"08cf40f5-5731-4996-b824-bb2975c3eb4f","Type":"ContainerDied","Data":"5fda273dad65a5ca1f904dd87005205b74572be97076dbc6ae4c9a621cf4baad"}
Sep 10 23:13:58.763858 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:58.763530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd" event={"ID":"08cf40f5-5731-4996-b824-bb2975c3eb4f","Type":"ContainerStarted","Data":"3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1"}
Sep 10 23:13:59.793151 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.793091    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:13:59.796886 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.796863    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.801544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.801367    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3a438f\"/\"default-dockercfg-c2qcj\""
Sep 10 23:13:59.801544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.801381    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-3a438f\"/\"autores-model-3a438f-kserve-self-signed-certs\""
Sep 10 23:13:59.801544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.801387    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3a438f\"/\"openshift-service-ca.crt\""
Sep 10 23:13:59.801544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.801450    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-3a438f\"/\"kube-root-ca.crt\""
Sep 10 23:13:59.810463 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.810439    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:13:59.910102 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.910081    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:13:59.915332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915310    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.915420 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915366    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.915464 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915454    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.915506 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915483    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.915547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915529    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:13:59.915586 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:13:59.915555    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjhhr\" (UniqueName: \"kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016313 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016255    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca\") pod \"08cf40f5-5731-4996-b824-bb2975c3eb4f\" (UID: \"08cf40f5-5731-4996-b824-bb2975c3eb4f\") "
Sep 10 23:14:00.016510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016421    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016472    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016492    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016516    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjhhr\" (UniqueName: \"kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.016693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016648    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08cf40f5-5731-4996-b824-bb2975c3eb4f" (UID: "08cf40f5-5731-4996-b824-bb2975c3eb4f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:14:00.016990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016972    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.017060 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.016996    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.017118 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.017062    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.018920 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.018853    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.019189 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.019168    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.026313 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.026271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjhhr\" (UniqueName: \"kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr\") pod \"autores-model-3a438f-kserve-574b5df49f-shsn9\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") " pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.109066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.109030    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:00.124315 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.118513    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08cf40f5-5731-4996-b824-bb2975c3eb4f-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:14:00.389254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.389228    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:14:00.391248 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:14:00.391208    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb6968aa_730c_4fcb_b05c_d2fe6b49bf25.slice/crio-12be77e900c5aff8cb5ffe0144e40d9bd228e7f69fcff225c484876f61f05dda WatchSource:0}: Error finding container 12be77e900c5aff8cb5ffe0144e40d9bd228e7f69fcff225c484876f61f05dda: Status 404 returned error can't find the container with id 12be77e900c5aff8cb5ffe0144e40d9bd228e7f69fcff225c484876f61f05dda
Sep 10 23:14:00.775800 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.775747    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" event={"ID":"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25","Type":"ContainerStarted","Data":"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"}
Sep 10 23:14:00.775968 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.775806    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" event={"ID":"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25","Type":"ContainerStarted","Data":"12be77e900c5aff8cb5ffe0144e40d9bd228e7f69fcff225c484876f61f05dda"}
Sep 10 23:14:00.777088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.777065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd" event={"ID":"08cf40f5-5731-4996-b824-bb2975c3eb4f","Type":"ContainerDied","Data":"3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1"}
Sep 10 23:14:00.777088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.777088    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"
Sep 10 23:14:00.777358 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.777103    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3206e10fc540cde778e3171eab250577441ee07441e36c91b8bb8d77f01b70b1"
Sep 10 23:14:00.794889 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:00.794847    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" podStartSLOduration=1.794831997 podStartE2EDuration="1.794831997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:14:00.794060312 +0000 UTC m=+963.080477260" watchObservedRunningTime="2026-09-10 23:14:00.794831997 +0000 UTC m=+963.081248960"
Sep 10 23:14:03.963892 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:03.963856    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"]
Sep 10 23:14:03.966804 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:03.966777    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-00575c78-zwzzd"]
Sep 10 23:14:04.209581 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:04.209543    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08cf40f5-5731-4996-b824-bb2975c3eb4f" path="/var/lib/kubelet/pods/08cf40f5-5731-4996-b824-bb2975c3eb4f/volumes"
Sep 10 23:14:04.466625 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:04.466588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:14:10.109342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:10.109286    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:10.109744 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:10.109370    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:10.121950 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:10.121914    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:10.838821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:10.838796    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:14:28.059945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:28.059844    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:14:33.709912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.709871    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"]
Sep 10 23:14:33.710447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.710338    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08cf40f5-5731-4996-b824-bb2975c3eb4f" containerName="revoke-keys"
Sep 10 23:14:33.710447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.710352    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="08cf40f5-5731-4996-b824-bb2975c3eb4f" containerName="revoke-keys"
Sep 10 23:14:33.710447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.710433    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="08cf40f5-5731-4996-b824-bb2975c3eb4f" containerName="revoke-keys"
Sep 10 23:14:33.713779 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.713763    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:33.716106 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.716081    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:14:33.716238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.716118    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:14:33.720978 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.720951    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"]
Sep 10 23:14:33.837238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.837206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd\" (UID: \"349b4a6d-a297-43e0-89ae-bf15ca3b0373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:33.938729 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.938694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd\" (UID: \"349b4a6d-a297-43e0-89ae-bf15ca3b0373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:33.939340 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:33.939321    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd\" (UID: \"349b4a6d-a297-43e0-89ae-bf15ca3b0373\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:34.029701 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:34.029614    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:34.303244 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:34.303169    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"]
Sep 10 23:14:34.306502 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:14:34.306477    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349b4a6d_a297_43e0_89ae_bf15ca3b0373.slice/crio-b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504 WatchSource:0}: Error finding container b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504: Status 404 returned error can't find the container with id b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504
Sep 10 23:14:34.953438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:34.953407    2565 generic.go:356] "Generic (PLEG): container finished" podID="349b4a6d-a297-43e0-89ae-bf15ca3b0373" containerID="1c5acca67e40e4e4128078eb036a7dace27845ba2fd7e8b53ae55d62a274b006" exitCode=0
Sep 10 23:14:34.953826 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:34.953496    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd" event={"ID":"349b4a6d-a297-43e0-89ae-bf15ca3b0373","Type":"ContainerDied","Data":"1c5acca67e40e4e4128078eb036a7dace27845ba2fd7e8b53ae55d62a274b006"}
Sep 10 23:14:34.953826 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:34.953563    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd" event={"ID":"349b4a6d-a297-43e0-89ae-bf15ca3b0373","Type":"ContainerStarted","Data":"b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504"}
Sep 10 23:14:36.088014 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.087986    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:36.262273 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.262184    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca\") pod \"349b4a6d-a297-43e0-89ae-bf15ca3b0373\" (UID: \"349b4a6d-a297-43e0-89ae-bf15ca3b0373\") "
Sep 10 23:14:36.262626 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.262603    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "349b4a6d-a297-43e0-89ae-bf15ca3b0373" (UID: "349b4a6d-a297-43e0-89ae-bf15ca3b0373"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:14:36.363643 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.363610    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/349b4a6d-a297-43e0-89ae-bf15ca3b0373-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:14:36.964897 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.964864    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"
Sep 10 23:14:36.964897 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.964892    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd" event={"ID":"349b4a6d-a297-43e0-89ae-bf15ca3b0373","Type":"ContainerDied","Data":"b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504"}
Sep 10 23:14:36.965109 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:36.964941    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1b35c115fd7f6b1b3b0acebf70a1656796a94c51c5d7e7238e1c3d00035d504"
Sep 10 23:14:41.143360 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:41.143318    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"]
Sep 10 23:14:41.146034 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:41.146010    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-e5ec8ad4-hv8rd"]
Sep 10 23:14:42.212200 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:42.212157    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="349b4a6d-a297-43e0-89ae-bf15ca3b0373" path="/var/lib/kubelet/pods/349b4a6d-a297-43e0-89ae-bf15ca3b0373/volumes"
Sep 10 23:14:47.361696 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:47.361660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:14:50.791227 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:14:50.791194    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:15:00.147736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.147698    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"]
Sep 10 23:15:00.148159 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.148115    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349b4a6d-a297-43e0-89ae-bf15ca3b0373" containerName="revoke-keys"
Sep 10 23:15:00.148159 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.148126    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="349b4a6d-a297-43e0-89ae-bf15ca3b0373" containerName="revoke-keys"
Sep 10 23:15:00.148247 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.148196    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="349b4a6d-a297-43e0-89ae-bf15ca3b0373" containerName="revoke-keys"
Sep 10 23:15:00.151460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.151439    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"
Sep 10 23:15:00.153907 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.153882    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:15:00.160006 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.159559    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"]
Sep 10 23:15:00.164071 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.164048    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"
Sep 10 23:15:00.441866 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:00.441840    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"]
Sep 10 23:15:00.444642 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:15:00.444613    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf182c350_78b4_49ed_9845_70d8ede84c85.slice/crio-df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57 WatchSource:0}: Error finding container df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57: Status 404 returned error can't find the container with id df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57
Sep 10 23:15:01.091485 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:01.091450    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb" event={"ID":"f182c350-78b4-49ed-9845-70d8ede84c85","Type":"ContainerStarted","Data":"1d32cedf5cab6c821c81894865fec1e360cda3c85d11c7703bce86e58e081df2"}
Sep 10 23:15:01.091653 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:01.091513    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb" event={"ID":"f182c350-78b4-49ed-9845-70d8ede84c85","Type":"ContainerStarted","Data":"df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57"}
Sep 10 23:15:01.110078 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:01.110026    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb" podStartSLOduration=1.110011248 podStartE2EDuration="1.110011248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:15:01.108184532 +0000 UTC m=+1023.394601480" watchObservedRunningTime="2026-09-10 23:15:01.110011248 +0000 UTC m=+1023.396428193"
Sep 10 23:15:02.098476 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:02.098439    2565 generic.go:356] "Generic (PLEG): container finished" podID="f182c350-78b4-49ed-9845-70d8ede84c85" containerID="1d32cedf5cab6c821c81894865fec1e360cda3c85d11c7703bce86e58e081df2" exitCode=0
Sep 10 23:15:02.098845 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:02.098486    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb" event={"ID":"f182c350-78b4-49ed-9845-70d8ede84c85","Type":"ContainerDied","Data":"1d32cedf5cab6c821c81894865fec1e360cda3c85d11c7703bce86e58e081df2"}
Sep 10 23:15:03.235637 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:03.235612    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"
Sep 10 23:15:03.962617 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:03.962582    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:15:04.109263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:04.109233    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"
Sep 10 23:15:04.109263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:04.109243    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb" event={"ID":"f182c350-78b4-49ed-9845-70d8ede84c85","Type":"ContainerDied","Data":"df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57"}
Sep 10 23:15:04.109530 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:04.109301    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df72604a16c79192ad0fd75d446fc759e3523e7eca2015eebec134ccca5fdd57"
Sep 10 23:15:05.812479 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:05.812428    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" podUID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" containerName="main" containerID="cri-o://dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01" gracePeriod=45
Sep 10 23:15:06.062738 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.062674    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:15:06.121932 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.121895    2565 generic.go:356] "Generic (PLEG): container finished" podID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" containerID="dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01" exitCode=0
Sep 10 23:15:06.121932 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.121934    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"
Sep 10 23:15:06.122129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.121965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" event={"ID":"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25","Type":"ContainerDied","Data":"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"}
Sep 10 23:15:06.122129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.122006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9" event={"ID":"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25","Type":"ContainerDied","Data":"12be77e900c5aff8cb5ffe0144e40d9bd228e7f69fcff225c484876f61f05dda"}
Sep 10 23:15:06.122129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.122025    2565 scope.go:117] "RemoveContainer" containerID="dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"
Sep 10 23:15:06.130756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.130736    2565 scope.go:117] "RemoveContainer" containerID="dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"
Sep 10 23:15:06.131043 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:15:06.131026    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01\": container with ID starting with dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01 not found: ID does not exist" containerID="dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"
Sep 10 23:15:06.131105 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.131050    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01"} err="failed to get container status \"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01\": rpc error: code = NotFound desc = could not find container \"dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01\": container with ID starting with dbb47cef2cc5e2e1530104c5a3a596517a2be58faafb834238be76e8f5fc1d01 not found: ID does not exist"
Sep 10 23:15:06.163590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163559    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.163590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163589    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.163835 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163611    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.163835 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163660    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.163835 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163709    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjhhr\" (UniqueName: \"kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.163835 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163758    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs\") pod \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\" (UID: \"eb6968aa-730c-4fcb-b05c-d2fe6b49bf25\") "
Sep 10 23:15:06.164096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163846    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:06.164096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.163859    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home" (OuterVolumeSpecName: "home") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:06.164096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.164020    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:06.164263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.164133    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.164263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.164152    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.164263 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.164166    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.166075 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.166047    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:15:06.166183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.166091    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr" (OuterVolumeSpecName: "kube-api-access-gjhhr") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "kube-api-access-gjhhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:15:06.166183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.166154    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm" (OuterVolumeSpecName: "dshm") pod "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" (UID: "eb6968aa-730c-4fcb-b05c-d2fe6b49bf25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:06.265684 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.265651    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.265684 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.265680    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjhhr\" (UniqueName: \"kubernetes.io/projected/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-kube-api-access-gjhhr\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.265684 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.265691    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:06.437655 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.437614    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:15:06.444455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:06.444430    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-3a438f/autores-model-3a438f-kserve-574b5df49f-shsn9"]
Sep 10 23:15:08.209656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:08.209623    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" path="/var/lib/kubelet/pods/eb6968aa-730c-4fcb-b05c-d2fe6b49bf25/volumes"
Sep 10 23:15:20.365849 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.365810    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:15:20.366212 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366205    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f182c350-78b4-49ed-9845-70d8ede84c85" containerName="cleanup"
Sep 10 23:15:20.366254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366216    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f182c350-78b4-49ed-9845-70d8ede84c85" containerName="cleanup"
Sep 10 23:15:20.366254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366236    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" containerName="main"
Sep 10 23:15:20.366254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366241    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" containerName="main"
Sep 10 23:15:20.366539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366352    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb6968aa-730c-4fcb-b05c-d2fe6b49bf25" containerName="main"
Sep 10 23:15:20.366539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.366365    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f182c350-78b4-49ed-9845-70d8ede84c85" containerName="cleanup"
Sep 10 23:15:20.370520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.370504    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.373340 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.373319    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-13cbbb71-kserve-self-signed-certs\""
Sep 10 23:15:20.380087 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.380065    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:15:20.409280 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409243    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.409482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.409482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409352    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.409482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn8qc\" (UniqueName: \"kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.409482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.409645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.409500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510628 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510576    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn8qc\" (UniqueName: \"kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.510831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.510814    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.511177 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.511086    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.511177 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.511120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.511177 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.511130    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.513211 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.513179    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.513422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.513405    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.520963 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.520942    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn8qc\" (UniqueName: \"kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc\") pod \"e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") " pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.680058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.680028    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:20.954924 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:20.954898    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:15:20.957607 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:15:20.957576    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc5424ca_de61_4540_a1fc_591cf39ffa97.slice/crio-a02d4d64bc2a2ddd057f286b8ebca7ca0986f449912ba9d46905c64377dcabbf WatchSource:0}: Error finding container a02d4d64bc2a2ddd057f286b8ebca7ca0986f449912ba9d46905c64377dcabbf: Status 404 returned error can't find the container with id a02d4d64bc2a2ddd057f286b8ebca7ca0986f449912ba9d46905c64377dcabbf
Sep 10 23:15:21.204594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:21.204492    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" event={"ID":"bc5424ca-de61-4540-a1fc-591cf39ffa97","Type":"ContainerStarted","Data":"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"}
Sep 10 23:15:21.204594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:21.204543    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" event={"ID":"bc5424ca-de61-4540-a1fc-591cf39ffa97","Type":"ContainerStarted","Data":"a02d4d64bc2a2ddd057f286b8ebca7ca0986f449912ba9d46905c64377dcabbf"}
Sep 10 23:15:24.957951 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:24.957891    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" podStartSLOduration=4.957873023 podStartE2EDuration="4.957873023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:15:21.221573326 +0000 UTC m=+1043.507990273" watchObservedRunningTime="2026-09-10 23:15:24.957873023 +0000 UTC m=+1047.244289971"
Sep 10 23:15:24.959440 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:24.959418    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:15:30.680891 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:30.680856    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:30.681422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:30.680933    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:30.693063 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:30.693038    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:31.265032 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:31.264999    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:15:37.536139 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:37.536108    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:15:39.670056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:39.670018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:15:52.555944 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.555834    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" podUID="89396197-cf5a-446e-a52d-70767fb65176" containerName="main" containerID="cri-o://07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc" gracePeriod=45
Sep 10 23:15:52.817710 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.817650    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:15:52.832233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832209    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832354 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832253    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832354 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832273    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832354 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832336    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxqtg\" (UniqueName: \"kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832523 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832384    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832523 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832469    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir\") pod \"89396197-cf5a-446e-a52d-70767fb65176\" (UID: \"89396197-cf5a-446e-a52d-70767fb65176\") "
Sep 10 23:15:52.832627 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832520    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache" (OuterVolumeSpecName: "model-cache") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:52.832627 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832536    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home" (OuterVolumeSpecName: "home") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:52.832776 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832755    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:52.832776 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832773    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:52.832861 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.832786    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:52.834729 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.834704    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:15:52.834729 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.834713    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm" (OuterVolumeSpecName: "dshm") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:15:52.834946 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.834928    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg" (OuterVolumeSpecName: "kube-api-access-fxqtg") pod "89396197-cf5a-446e-a52d-70767fb65176" (UID: "89396197-cf5a-446e-a52d-70767fb65176"). InnerVolumeSpecName "kube-api-access-fxqtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:15:52.933876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.933842    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:52.933876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.933869    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxqtg\" (UniqueName: \"kubernetes.io/projected/89396197-cf5a-446e-a52d-70767fb65176-kube-api-access-fxqtg\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:52.933876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.933880    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89396197-cf5a-446e-a52d-70767fb65176-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:52.933876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:52.933887    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89396197-cf5a-446e-a52d-70767fb65176-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:15:53.375258 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.375223    2565 generic.go:356] "Generic (PLEG): container finished" podID="89396197-cf5a-446e-a52d-70767fb65176" containerID="07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc" exitCode=0
Sep 10 23:15:53.375460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.375268    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"
Sep 10 23:15:53.375460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.375324    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" event={"ID":"89396197-cf5a-446e-a52d-70767fb65176","Type":"ContainerDied","Data":"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"}
Sep 10 23:15:53.375460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.375370    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb" event={"ID":"89396197-cf5a-446e-a52d-70767fb65176","Type":"ContainerDied","Data":"7ca460baa9de43b4c77e88b9c84f4d8b6b4a843e68126f3bd2010153a2d691bf"}
Sep 10 23:15:53.375460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.375392    2565 scope.go:117] "RemoveContainer" containerID="07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"
Sep 10 23:15:53.384648 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.384630    2565 scope.go:117] "RemoveContainer" containerID="07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"
Sep 10 23:15:53.384924 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:15:53.384904    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc\": container with ID starting with 07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc not found: ID does not exist" containerID="07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"
Sep 10 23:15:53.384991 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.384936    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc"} err="failed to get container status \"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc\": rpc error: code = NotFound desc = could not find container \"07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc\": container with ID starting with 07ec971e70e8902452bd9a79ec1da6305cebbf7767ae03b9273e9376d576c5dc not found: ID does not exist"
Sep 10 23:15:53.396677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.396653    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:15:53.401408 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:53.401384    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-13cbbb71-kserve-6dc89fbd7d-dr7tb"]
Sep 10 23:15:54.210956 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:54.210924    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89396197-cf5a-446e-a52d-70767fb65176" path="/var/lib/kubelet/pods/89396197-cf5a-446e-a52d-70767fb65176/volumes"
Sep 10 23:15:55.478331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:15:55.478271    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:16:05.664261 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.664229    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:16:05.664679 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.664664    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89396197-cf5a-446e-a52d-70767fb65176" containerName="main"
Sep 10 23:16:05.664722 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.664680    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="89396197-cf5a-446e-a52d-70767fb65176" containerName="main"
Sep 10 23:16:05.664757 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.664736    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="89396197-cf5a-446e-a52d-70767fb65176" containerName="main"
Sep 10 23:16:05.669221 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.669201    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.671645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.671624    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1f1f85\"/\"kube-root-ca.crt\""
Sep 10 23:16:05.672428 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.672399    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1f1f85\"/\"default-dockercfg-c2px7\""
Sep 10 23:16:05.672550 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.672448    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1f1f85\"/\"test-model-1f1f85-kserve-self-signed-certs\""
Sep 10 23:16:05.672550 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.672446    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1f1f85\"/\"openshift-service-ca.crt\""
Sep 10 23:16:05.681084 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.681060    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:16:05.751943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.751910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.752120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.751959    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7znc2\" (UniqueName: \"kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.752120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.752042    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.752120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.752081    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.752120 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.752111    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.752253 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.752174    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.852844 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852811    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.852844 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7znc2\" (UniqueName: \"kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853058 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.852962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853309 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.853254    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853380 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.853322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.853380 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.853371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.855394 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.855372    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.855560 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.855543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.862081 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.862056    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7znc2\" (UniqueName: \"kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2\") pod \"test-model-1f1f85-kserve-688cfcb775-htzh9\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") " pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:05.978517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:05.978479    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:06.250939 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:06.250912    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:16:06.252980 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:16:06.252947    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef329104_bdcf_4e41_8220_797266feb339.slice/crio-5c4d2a89fd553eb67b25aee2f26d774ffc699cb87ad2ef902dc68c68d03f9633 WatchSource:0}: Error finding container 5c4d2a89fd553eb67b25aee2f26d774ffc699cb87ad2ef902dc68c68d03f9633: Status 404 returned error can't find the container with id 5c4d2a89fd553eb67b25aee2f26d774ffc699cb87ad2ef902dc68c68d03f9633
Sep 10 23:16:06.445992 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:06.445959    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" event={"ID":"ef329104-bdcf-4e41-8220-797266feb339","Type":"ContainerStarted","Data":"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"}
Sep 10 23:16:06.445992 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:06.446006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" event={"ID":"ef329104-bdcf-4e41-8220-797266feb339","Type":"ContainerStarted","Data":"5c4d2a89fd553eb67b25aee2f26d774ffc699cb87ad2ef902dc68c68d03f9633"}
Sep 10 23:16:06.465490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:06.465444    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" podStartSLOduration=1.4654290269999999 podStartE2EDuration="1.465429027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:16:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:16:06.464561138 +0000 UTC m=+1088.750978085" watchObservedRunningTime="2026-09-10 23:16:06.465429027 +0000 UTC m=+1088.751846046"
Sep 10 23:16:07.812437 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:07.812400    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:16:09.948660 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.948626    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:09.952403 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.952385    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.954525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.954502    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-8972e2\"/\"kube-root-ca.crt\""
Sep 10 23:16:09.954655 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.954540    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-8972e2\"/\"explicit-model-8972e2-kserve-self-signed-certs\""
Sep 10 23:16:09.955338 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.955316    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-8972e2\"/\"openshift-service-ca.crt\""
Sep 10 23:16:09.955450 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.955317    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-8972e2\"/\"default-dockercfg-kvzr7\""
Sep 10 23:16:09.962095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.962073    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:09.994374 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994349    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m8lz\" (UniqueName: \"kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.994524 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994410    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.994524 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.994524 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.994524 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:09.994524 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:09.994484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095531 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8lz\" (UniqueName: \"kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095577    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095645    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.095680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.096021 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.095997    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.096086 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.096023    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.096086 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.096042    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.098130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.098105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.098418 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.098401    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.105188 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.105163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8lz\" (UniqueName: \"kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz\") pod \"explicit-model-8972e2-kserve-54645bd5ff-hw28t\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") " pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.262987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.262901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:10.541978 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:10.541883    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:10.545675 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:16:10.545645    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8986b3c8_3b67_45f8_9438_cf4136c099e1.slice/crio-635c81a00015cfdad71d406c3d1c89d6d7f63ff1f97a2c9746bd0912bad32278 WatchSource:0}: Error finding container 635c81a00015cfdad71d406c3d1c89d6d7f63ff1f97a2c9746bd0912bad32278: Status 404 returned error can't find the container with id 635c81a00015cfdad71d406c3d1c89d6d7f63ff1f97a2c9746bd0912bad32278
Sep 10 23:16:11.364766 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:11.364724    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:16:11.472855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:11.472817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" event={"ID":"8986b3c8-3b67-45f8-9438-cf4136c099e1","Type":"ContainerStarted","Data":"8b812d265b5ada367e42d4afbdc09674f8bac52dc0bb7d05800875ab1b3b2711"}
Sep 10 23:16:11.473025 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:11.472870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" event={"ID":"8986b3c8-3b67-45f8-9438-cf4136c099e1","Type":"ContainerStarted","Data":"635c81a00015cfdad71d406c3d1c89d6d7f63ff1f97a2c9746bd0912bad32278"}
Sep 10 23:16:11.511726 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:11.511669    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" podStartSLOduration=2.511651545 podStartE2EDuration="2.511651545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:16:11.508609704 +0000 UTC m=+1093.795026650" watchObservedRunningTime="2026-09-10 23:16:11.511651545 +0000 UTC m=+1093.798068491"
Sep 10 23:16:15.979363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:15.979323    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:15.979363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:15.979377    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:15.991672 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:15.991648    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:16.509579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:16.509552    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:16:18.420561 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:18.420532    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:20.263735 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:20.263693    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:22.831585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:22.831511    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" podUID="bc5424ca-de61-4540-a1fc-591cf39ffa97" containerName="main" containerID="cri-o://c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4" gracePeriod=45
Sep 10 23:16:23.076272 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.076249    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:16:23.224098 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224070    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224134    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn8qc\" (UniqueName: \"kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224234    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224255    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224270    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm\") pod \"bc5424ca-de61-4540-a1fc-591cf39ffa97\" (UID: \"bc5424ca-de61-4540-a1fc-591cf39ffa97\") "
Sep 10 23:16:23.224590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224400    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:23.224590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224526    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache" (OuterVolumeSpecName: "model-cache") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:23.224590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224532    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home" (OuterVolumeSpecName: "home") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:23.224713 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224622    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.224713 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224640    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.224713 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.224657    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.226615 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.226571    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:16:23.226730 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.226710    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc" (OuterVolumeSpecName: "kube-api-access-jn8qc") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "kube-api-access-jn8qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:16:23.226840 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.226816    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm" (OuterVolumeSpecName: "dshm") pod "bc5424ca-de61-4540-a1fc-591cf39ffa97" (UID: "bc5424ca-de61-4540-a1fc-591cf39ffa97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:23.326110 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.326078    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn8qc\" (UniqueName: \"kubernetes.io/projected/bc5424ca-de61-4540-a1fc-591cf39ffa97-kube-api-access-jn8qc\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.326110 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.326103    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc5424ca-de61-4540-a1fc-591cf39ffa97-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.326110 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.326113    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bc5424ca-de61-4540-a1fc-591cf39ffa97-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:23.543600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.543519    2565 generic.go:356] "Generic (PLEG): container finished" podID="bc5424ca-de61-4540-a1fc-591cf39ffa97" containerID="c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4" exitCode=0
Sep 10 23:16:23.543600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.543570    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"
Sep 10 23:16:23.543600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.543602    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" event={"ID":"bc5424ca-de61-4540-a1fc-591cf39ffa97","Type":"ContainerDied","Data":"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"}
Sep 10 23:16:23.543833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.543644    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt" event={"ID":"bc5424ca-de61-4540-a1fc-591cf39ffa97","Type":"ContainerDied","Data":"a02d4d64bc2a2ddd057f286b8ebca7ca0986f449912ba9d46905c64377dcabbf"}
Sep 10 23:16:23.543833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.543663    2565 scope.go:117] "RemoveContainer" containerID="c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"
Sep 10 23:16:23.554009 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.553991    2565 scope.go:117] "RemoveContainer" containerID="c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"
Sep 10 23:16:23.554310 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:16:23.554269    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4\": container with ID starting with c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4 not found: ID does not exist" containerID="c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"
Sep 10 23:16:23.554407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.554327    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4"} err="failed to get container status \"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4\": rpc error: code = NotFound desc = could not find container \"c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4\": container with ID starting with c2d4a7a820b4fd1eff18a45b59887375b6e6ae9152f258f60fba20685ac003d4 not found: ID does not exist"
Sep 10 23:16:23.577611 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.577586    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:16:23.583179 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:23.583152    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-13cbbb71-kserve-5c54d4f66d-lmzlt"]
Sep 10 23:16:24.209906 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:24.209871    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5424ca-de61-4540-a1fc-591cf39ffa97" path="/var/lib/kubelet/pods/bc5424ca-de61-4540-a1fc-591cf39ffa97/volumes"
Sep 10 23:16:32.662213 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:32.662173    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:16:33.440570 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.440531    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" podUID="8986b3c8-3b67-45f8-9438-cf4136c099e1" containerName="main" containerID="cri-o://8b812d265b5ada367e42d4afbdc09674f8bac52dc0bb7d05800875ab1b3b2711" gracePeriod=45
Sep 10 23:16:33.604990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.604957    2565 generic.go:356] "Generic (PLEG): container finished" podID="8986b3c8-3b67-45f8-9438-cf4136c099e1" containerID="8b812d265b5ada367e42d4afbdc09674f8bac52dc0bb7d05800875ab1b3b2711" exitCode=0
Sep 10 23:16:33.605213 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.605025    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" event={"ID":"8986b3c8-3b67-45f8-9438-cf4136c099e1","Type":"ContainerDied","Data":"8b812d265b5ada367e42d4afbdc09674f8bac52dc0bb7d05800875ab1b3b2711"}
Sep 10 23:16:33.700868 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.700805    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:33.731132 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731105    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731307 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731162    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731307 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731219    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731307 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731247    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m8lz\" (UniqueName: \"kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731307 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731270    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731535 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731385    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm\") pod \"8986b3c8-3b67-45f8-9438-cf4136c099e1\" (UID: \"8986b3c8-3b67-45f8-9438-cf4136c099e1\") "
Sep 10 23:16:33.731535 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731417    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:33.731535 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731504    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home" (OuterVolumeSpecName: "home") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:33.731684 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731556    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:33.731808 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731786    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:33.731865 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731818    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:33.731865 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.731833    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:33.733652 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.733624    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz" (OuterVolumeSpecName: "kube-api-access-8m8lz") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "kube-api-access-8m8lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:16:33.733750 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.733726    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:16:33.733948 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.733930    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm" (OuterVolumeSpecName: "dshm") pod "8986b3c8-3b67-45f8-9438-cf4136c099e1" (UID: "8986b3c8-3b67-45f8-9438-cf4136c099e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:16:33.832320 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.832257    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m8lz\" (UniqueName: \"kubernetes.io/projected/8986b3c8-3b67-45f8-9438-cf4136c099e1-kube-api-access-8m8lz\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:33.832320 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.832321    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8986b3c8-3b67-45f8-9438-cf4136c099e1-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:33.832528 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:33.832333    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8986b3c8-3b67-45f8-9438-cf4136c099e1-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:16:34.618218 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:34.618125    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t" event={"ID":"8986b3c8-3b67-45f8-9438-cf4136c099e1","Type":"ContainerDied","Data":"635c81a00015cfdad71d406c3d1c89d6d7f63ff1f97a2c9746bd0912bad32278"}
Sep 10 23:16:34.618218 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:34.618161    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"
Sep 10 23:16:34.618218 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:34.618187    2565 scope.go:117] "RemoveContainer" containerID="8b812d265b5ada367e42d4afbdc09674f8bac52dc0bb7d05800875ab1b3b2711"
Sep 10 23:16:34.636968 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:34.636935    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:34.643209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:34.643183    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-8972e2/explicit-model-8972e2-kserve-54645bd5ff-hw28t"]
Sep 10 23:16:36.212173 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:36.212136    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8986b3c8-3b67-45f8-9438-cf4136c099e1" path="/var/lib/kubelet/pods/8986b3c8-3b67-45f8-9438-cf4136c099e1/volumes"
Sep 10 23:16:55.357090 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:16:55.357049    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:17:12.252300 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:12.252249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:17:24.144691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.144603    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:24.145249 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145226    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5424ca-de61-4540-a1fc-591cf39ffa97" containerName="main"
Sep 10 23:17:24.145314 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145256    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5424ca-de61-4540-a1fc-591cf39ffa97" containerName="main"
Sep 10 23:17:24.145356 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145326    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8986b3c8-3b67-45f8-9438-cf4136c099e1" containerName="main"
Sep 10 23:17:24.145356 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145335    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8986b3c8-3b67-45f8-9438-cf4136c099e1" containerName="main"
Sep 10 23:17:24.145456 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145443    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8986b3c8-3b67-45f8-9438-cf4136c099e1" containerName="main"
Sep 10 23:17:24.145499 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.145468    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5424ca-de61-4540-a1fc-591cf39ffa97" containerName="main"
Sep 10 23:17:24.150520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.150499    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.152878 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.152855    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-5ea2db\"/\"kube-root-ca.crt\""
Sep 10 23:17:24.153008 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.152855    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-5ea2db\"/\"openshift-service-ca.crt\""
Sep 10 23:17:24.153745 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.153726    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-5ea2db\"/\"noten-model-5ea2db-kserve-self-signed-certs\""
Sep 10 23:17:24.154142 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.153726    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-5ea2db\"/\"default-dockercfg-b8n2t\""
Sep 10 23:17:24.157244 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.157219    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:24.204884 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.204845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.205064 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.204956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.205064 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.204996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ffv2\" (UniqueName: \"kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.205064 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.205051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.205241 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.205128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.205241 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.205181    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306304 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ffv2\" (UniqueName: \"kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306323    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306352    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306377    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306401    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306860 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.306860 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.306845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.308922 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.308897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.309153 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.309139    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.315626 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.315605    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ffv2\" (UniqueName: \"kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2\") pod \"noten-model-5ea2db-kserve-79f898755f-jzffl\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") " pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.461909 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.461873    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:24.738505 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.738481    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:24.740581 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:17:24.740552    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e216f2f_56b1_4758_b78f_92fb2115b0f6.slice/crio-f60f05103ecc4ab039119d1684803e01c97e97b1fc8f87f2bee5c5aa39387b98 WatchSource:0}: Error finding container f60f05103ecc4ab039119d1684803e01c97e97b1fc8f87f2bee5c5aa39387b98: Status 404 returned error can't find the container with id f60f05103ecc4ab039119d1684803e01c97e97b1fc8f87f2bee5c5aa39387b98
Sep 10 23:17:24.887142 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.887106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" event={"ID":"7e216f2f-56b1-4758-b78f-92fb2115b0f6","Type":"ContainerStarted","Data":"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"}
Sep 10 23:17:24.887325 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.887153    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" event={"ID":"7e216f2f-56b1-4758-b78f-92fb2115b0f6","Type":"ContainerStarted","Data":"f60f05103ecc4ab039119d1684803e01c97e97b1fc8f87f2bee5c5aa39387b98"}
Sep 10 23:17:24.905941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:24.905875    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" podStartSLOduration=0.90585817 podStartE2EDuration="905.85817ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:17:24.903363841 +0000 UTC m=+1167.189780783" watchObservedRunningTime="2026-09-10 23:17:24.90585817 +0000 UTC m=+1167.192275116"
Sep 10 23:17:26.560841 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:26.560801    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:17:27.447607 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:27.447577    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:28.646018 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:28.645982    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:17:34.462130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:34.462093    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:37.260101 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:37.260063    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:17:39.485907 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:39.485871    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"]
Sep 10 23:17:39.490783 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:39.490756    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-28dbae-29818035-8bvtb"]
Sep 10 23:17:40.210269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:40.210229    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f182c350-78b4-49ed-9845-70d8ede84c85" path="/var/lib/kubelet/pods/f182c350-78b4-49ed-9845-70d8ede84c85/volumes"
Sep 10 23:17:42.468578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.468514    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" podUID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" containerName="main" containerID="cri-o://7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f" gracePeriod=45
Sep 10 23:17:42.705594 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.705568    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:42.796178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796087    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796158    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796190    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796285    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ffv2\" (UniqueName: \"kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796345    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home\") pod \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\" (UID: \"7e216f2f-56b1-4758-b78f-92fb2115b0f6\") "
Sep 10 23:17:42.796455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796402    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:42.796703 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796574    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:42.796703 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796676    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home" (OuterVolumeSpecName: "home") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:42.796807 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796770    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.796807 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796787    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.796807 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.796800    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.798595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.798575    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm" (OuterVolumeSpecName: "dshm") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:42.798740 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.798717    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2" (OuterVolumeSpecName: "kube-api-access-8ffv2") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "kube-api-access-8ffv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:17:42.798740 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.798728    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e216f2f-56b1-4758-b78f-92fb2115b0f6" (UID: "7e216f2f-56b1-4758-b78f-92fb2115b0f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:17:42.898417 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.898384    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ffv2\" (UniqueName: \"kubernetes.io/projected/7e216f2f-56b1-4758-b78f-92fb2115b0f6-kube-api-access-8ffv2\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.898417 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.898413    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e216f2f-56b1-4758-b78f-92fb2115b0f6-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.898417 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.898422    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e216f2f-56b1-4758-b78f-92fb2115b0f6-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:42.988206 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.988168    2565 generic.go:356] "Generic (PLEG): container finished" podID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" containerID="7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f" exitCode=0
Sep 10 23:17:42.988407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.988232    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"
Sep 10 23:17:42.988407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.988259    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" event={"ID":"7e216f2f-56b1-4758-b78f-92fb2115b0f6","Type":"ContainerDied","Data":"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"}
Sep 10 23:17:42.988407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.988330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl" event={"ID":"7e216f2f-56b1-4758-b78f-92fb2115b0f6","Type":"ContainerDied","Data":"f60f05103ecc4ab039119d1684803e01c97e97b1fc8f87f2bee5c5aa39387b98"}
Sep 10 23:17:42.988407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.988350    2565 scope.go:117] "RemoveContainer" containerID="7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"
Sep 10 23:17:42.997805 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.997787    2565 scope.go:117] "RemoveContainer" containerID="7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"
Sep 10 23:17:42.998072 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:17:42.998051    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f\": container with ID starting with 7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f not found: ID does not exist" containerID="7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"
Sep 10 23:17:42.998118 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:42.998080    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f"} err="failed to get container status \"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f\": rpc error: code = NotFound desc = could not find container \"7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f\": container with ID starting with 7ea753267a04de153e0068064ce0fcf78571d8a656c9e61977ab02d434c9ba3f not found: ID does not exist"
Sep 10 23:17:43.009416 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.009389    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:43.013070 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.013050    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-5ea2db/noten-model-5ea2db-kserve-79f898755f-jzffl"]
Sep 10 23:17:43.666405 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.666366    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" podUID="ef329104-bdcf-4e41-8220-797266feb339" containerName="main" containerID="cri-o://4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d" gracePeriod=45
Sep 10 23:17:43.917554 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.917492    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:17:43.996562 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.996526    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef329104-bdcf-4e41-8220-797266feb339" containerID="4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d" exitCode=0
Sep 10 23:17:43.996562 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.996564    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"
Sep 10 23:17:43.996774 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.996611    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" event={"ID":"ef329104-bdcf-4e41-8220-797266feb339","Type":"ContainerDied","Data":"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"}
Sep 10 23:17:43.996774 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.996653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9" event={"ID":"ef329104-bdcf-4e41-8220-797266feb339","Type":"ContainerDied","Data":"5c4d2a89fd553eb67b25aee2f26d774ffc699cb87ad2ef902dc68c68d03f9633"}
Sep 10 23:17:43.996774 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:43.996672    2565 scope.go:117] "RemoveContainer" containerID="4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"
Sep 10 23:17:44.005179 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.005156    2565 scope.go:117] "RemoveContainer" containerID="4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"
Sep 10 23:17:44.005504 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:17:44.005472    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d\": container with ID starting with 4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d not found: ID does not exist" containerID="4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"
Sep 10 23:17:44.005606 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.005501    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d"} err="failed to get container status \"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d\": rpc error: code = NotFound desc = could not find container \"4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d\": container with ID starting with 4fdcd00b0b3aebd0a18607fda7ffc2ab45fa322feeb809eebee217efa8fb7f0d not found: ID does not exist"
Sep 10 23:17:44.007810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.007795    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.007868 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.007843    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.007912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.007877    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7znc2\" (UniqueName: \"kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.007960 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.007946    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.008003 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.007971    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.008056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.008017    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home\") pod \"ef329104-bdcf-4e41-8220-797266feb339\" (UID: \"ef329104-bdcf-4e41-8220-797266feb339\") "
Sep 10 23:17:44.008327 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.008244    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:44.008442 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.008330    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:44.008528 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.008506    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home" (OuterVolumeSpecName: "home") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:44.010343 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.010277    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2" (OuterVolumeSpecName: "kube-api-access-7znc2") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "kube-api-access-7znc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:17:44.010608 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.010590    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:17:44.010608 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.010594    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm" (OuterVolumeSpecName: "dshm") pod "ef329104-bdcf-4e41-8220-797266feb339" (UID: "ef329104-bdcf-4e41-8220-797266feb339"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:17:44.109444 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109408    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.109444 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109435    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef329104-bdcf-4e41-8220-797266feb339-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.109444 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109447    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7znc2\" (UniqueName: \"kubernetes.io/projected/ef329104-bdcf-4e41-8220-797266feb339-kube-api-access-7znc2\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.109676 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109476    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.109676 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109485    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.109676 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.109493    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef329104-bdcf-4e41-8220-797266feb339-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:17:44.210378 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.210336    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" path="/var/lib/kubelet/pods/7e216f2f-56b1-4758-b78f-92fb2115b0f6/volumes"
Sep 10 23:17:44.315134 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.315102    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:17:44.318869 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:44.318844    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1f1f85/test-model-1f1f85-kserve-688cfcb775-htzh9"]
Sep 10 23:17:46.209850 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:46.209815    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef329104-bdcf-4e41-8220-797266feb339" path="/var/lib/kubelet/pods/ef329104-bdcf-4e41-8220-797266feb339/volumes"
Sep 10 23:17:49.281539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:17:49.281498    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:18:00.060229 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:00.060191    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:18:14.761000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:14.760962    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:18:27.176823 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.176779    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7655-26bcf0"]
Sep 10 23:18:27.177205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177190    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" containerName="main"
Sep 10 23:18:27.177250 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177206    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" containerName="main"
Sep 10 23:18:27.177250 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177226    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef329104-bdcf-4e41-8220-797266feb339" containerName="main"
Sep 10 23:18:27.177250 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177231    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef329104-bdcf-4e41-8220-797266feb339" containerName="main"
Sep 10 23:18:27.177376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177320    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e216f2f-56b1-4758-b78f-92fb2115b0f6" containerName="main"
Sep 10 23:18:27.177376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.177331    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef329104-bdcf-4e41-8220-797266feb339" containerName="main"
Sep 10 23:18:27.180460 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.180441    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:27.182521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.182503    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nn896\""
Sep 10 23:18:27.187824 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.187663    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-26bcf0"]
Sep 10 23:18:27.320561 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.320521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrphl\" (UniqueName: \"kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl\") pod \"test-curl-7655-26bcf0\" (UID: \"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb\") " pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:27.423933 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.423893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrphl\" (UniqueName: \"kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl\") pod \"test-curl-7655-26bcf0\" (UID: \"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb\") " pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:27.432825 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.432761    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrphl\" (UniqueName: \"kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl\") pod \"test-curl-7655-26bcf0\" (UID: \"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb\") " pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:27.491112 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.491083    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:27.760159 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.760132    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-26bcf0"]
Sep 10 23:18:27.761962 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:18:27.761934    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3880b99b_5244_4bb8_9f93_a10bbc5bf0fb.slice/crio-dcb02581dac922cf2dcfd6d3a8b6567ff22bb598922be25f3af97552a3c71db9 WatchSource:0}: Error finding container dcb02581dac922cf2dcfd6d3a8b6567ff22bb598922be25f3af97552a3c71db9: Status 404 returned error can't find the container with id dcb02581dac922cf2dcfd6d3a8b6567ff22bb598922be25f3af97552a3c71db9
Sep 10 23:18:27.763751 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:27.763734    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:18:28.228375 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:28.228342    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-26bcf0" event={"ID":"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb","Type":"ContainerStarted","Data":"dcb02581dac922cf2dcfd6d3a8b6567ff22bb598922be25f3af97552a3c71db9"}
Sep 10 23:18:30.240869 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:30.240830    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-26bcf0" event={"ID":"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb","Type":"ContainerStarted","Data":"9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad"}
Sep 10 23:18:30.257180 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:30.257134    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7655-26bcf0" podStartSLOduration=1.160985169 podStartE2EDuration="3.257116995s" totalImagesPullingTime="2.096131826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:18:30.255371157 +0000 UTC m=+1232.541788104" watchObservedRunningTime="2026-09-10 23:18:30.257116995 +0000 UTC m=+1232.543533942"
Sep 10 23:18:30.889885 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:30.889850    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7655-26bcf0"]
Sep 10 23:18:30.893654 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:30.893624    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7655-26bcf0"]
Sep 10 23:18:31.001134 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.001101    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7655-d3e891"]
Sep 10 23:18:31.001516 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.001503    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" containerName="test-curl-7655-26bcf0"
Sep 10 23:18:31.001565 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.001518    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" containerName="test-curl-7655-26bcf0"
Sep 10 23:18:31.001605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.001586    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" containerName="test-curl-7655-26bcf0"
Sep 10 23:18:31.004717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.004699    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:31.011028 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.011002    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-d3e891"]
Sep 10 23:18:31.059431 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.059404    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhfpm\" (UniqueName: \"kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm\") pod \"test-curl-7655-d3e891\" (UID: \"c391ecfa-a9be-416f-ae45-98d453324ca6\") " pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:31.160790 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.160755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhfpm\" (UniqueName: \"kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm\") pod \"test-curl-7655-d3e891\" (UID: \"c391ecfa-a9be-416f-ae45-98d453324ca6\") " pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:31.168688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.168657    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhfpm\" (UniqueName: \"kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm\") pod \"test-curl-7655-d3e891\" (UID: \"c391ecfa-a9be-416f-ae45-98d453324ca6\") " pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:31.246119 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:31.246089    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7655-26bcf0" container="test-curl-7655-26bcf0"
Sep 10 23:18:31.315394 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.315277    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:31.585746 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:31.585721    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-d3e891"]
Sep 10 23:18:31.587912 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:18:31.587882    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc391ecfa_a9be_416f_ae45_98d453324ca6.slice/crio-297b13cdfc67f374e644a7271d686a775339379015d85789f998bacfb2b67b09 WatchSource:0}: Error finding container 297b13cdfc67f374e644a7271d686a775339379015d85789f998bacfb2b67b09: Status 404 returned error can't find the container with id 297b13cdfc67f374e644a7271d686a775339379015d85789f998bacfb2b67b09
Sep 10 23:18:32.252462 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.252422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-d3e891" event={"ID":"c391ecfa-a9be-416f-ae45-98d453324ca6","Type":"ContainerStarted","Data":"15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6"}
Sep 10 23:18:32.252894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.252472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-d3e891" event={"ID":"c391ecfa-a9be-416f-ae45-98d453324ca6","Type":"ContainerStarted","Data":"297b13cdfc67f374e644a7271d686a775339379015d85789f998bacfb2b67b09"}
Sep 10 23:18:32.252894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.252471    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7655-26bcf0" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" containerName="test-curl-7655-26bcf0" containerID="cri-o://9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad" gracePeriod=2
Sep 10 23:18:32.267906 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.267857    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7655-d3e891" podStartSLOduration=2.267843332 podStartE2EDuration="2.267843332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:18:32.265783596 +0000 UTC m=+1234.552200542" watchObservedRunningTime="2026-09-10 23:18:32.267843332 +0000 UTC m=+1234.554260339"
Sep 10 23:18:32.833658 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.833621    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7655-d3e891"]
Sep 10 23:18:32.837381 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.837348    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7655-d3e891"]
Sep 10 23:18:32.947079 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.947041    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7655-e71835"]
Sep 10 23:18:32.947489 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.947475    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" containerName="test-curl-7655-d3e891"
Sep 10 23:18:32.947534 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.947492    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" containerName="test-curl-7655-d3e891"
Sep 10 23:18:32.947583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.947573    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" containerName="test-curl-7655-d3e891"
Sep 10 23:18:32.950716 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.950700    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:32.956165 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.956139    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-e71835"]
Sep 10 23:18:32.977605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:32.977577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b86nx\" (UniqueName: \"kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx\") pod \"test-curl-7655-e71835\" (UID: \"d10fcfcc-ed6f-47cc-b562-6279f6769eb3\") " pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:33.078424 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:33.078385    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b86nx\" (UniqueName: \"kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx\") pod \"test-curl-7655-e71835\" (UID: \"d10fcfcc-ed6f-47cc-b562-6279f6769eb3\") " pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:33.086330 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:33.086279    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b86nx\" (UniqueName: \"kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx\") pod \"test-curl-7655-e71835\" (UID: \"d10fcfcc-ed6f-47cc-b562-6279f6769eb3\") " pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:33.257674 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:33.257597    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7655-d3e891" container="test-curl-7655-d3e891"
Sep 10 23:18:33.261410 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:33.261393    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:33.532169 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:33.532146    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-e71835"]
Sep 10 23:18:34.266046 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.266007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-e71835" event={"ID":"d10fcfcc-ed6f-47cc-b562-6279f6769eb3","Type":"ContainerStarted","Data":"50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed"}
Sep 10 23:18:34.266471 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.266063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-e71835" event={"ID":"d10fcfcc-ed6f-47cc-b562-6279f6769eb3","Type":"ContainerStarted","Data":"eac4428de3e987ee7cf5d27392baadb7a435681a6a78d23d775e731c16ca16c9"}
Sep 10 23:18:34.266471 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.266120    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7655-d3e891" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" containerName="test-curl-7655-d3e891" containerID="cri-o://15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6" gracePeriod=2
Sep 10 23:18:34.281695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.281642    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7655-e71835" podStartSLOduration=2.281624962 podStartE2EDuration="2.281624962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:18:34.279052121 +0000 UTC m=+1236.565469068" watchObservedRunningTime="2026-09-10 23:18:34.281624962 +0000 UTC m=+1236.568041908"
Sep 10 23:18:34.399536 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.399513    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:34.492626 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.492592    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrphl\" (UniqueName: \"kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl\") pod \"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb\" (UID: \"3880b99b-5244-4bb8-9f93-a10bbc5bf0fb\") "
Sep 10 23:18:34.494916 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.494883    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl" (OuterVolumeSpecName: "kube-api-access-zrphl") pod "3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" (UID: "3880b99b-5244-4bb8-9f93-a10bbc5bf0fb"). InnerVolumeSpecName "kube-api-access-zrphl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:18:34.594359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.594265    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrphl\" (UniqueName: \"kubernetes.io/projected/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb-kube-api-access-zrphl\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:18:34.857466 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.857376    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7655-e71835"]
Sep 10 23:18:34.859911 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:34.859882    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7655-e71835"]
Sep 10 23:18:35.273412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:35.273373    2565 generic.go:356] "Generic (PLEG): container finished" podID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" containerID="9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad" exitCode=137
Sep 10 23:18:35.273863 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:35.273421    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:35.273863 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:35.273474    2565 scope.go:117] "RemoveContainer" containerID="9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad"
Sep 10 23:18:35.273863 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:35.273693    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7655-e71835" container="test-curl-7655-e71835"
Sep 10 23:18:35.282413 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:35.282393    2565 scope.go:117] "RemoveContainer" containerID="9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad"
Sep 10 23:18:35.282696 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:35.282678    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad\": container with ID starting with 9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad not found: ID does not exist" containerID="9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad"
Sep 10 23:18:35.282785 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:35.282705    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad"} err="failed to get container status \"9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad\": rpc error: code = NotFound desc = could not find container \"9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad\": container with ID starting with 9b9ce704f3ed1125baa733ea540fdd935b8eaaf6c904ff1aaf6cd021cbfe19ad not found: ID does not exist"
Sep 10 23:18:35.283326 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:35.283282    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-26bcf0\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" pod="opendatahub/test-curl-7655-26bcf0"
Sep 10 23:18:36.210052 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.210013    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3880b99b-5244-4bb8-9f93-a10bbc5bf0fb" path="/var/lib/kubelet/pods/3880b99b-5244-4bb8-9f93-a10bbc5bf0fb/volumes"
Sep 10 23:18:36.279713 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.279673    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7655-e71835" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" containerName="test-curl-7655-e71835" containerID="cri-o://50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed" gracePeriod=2
Sep 10 23:18:36.441575 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.441550    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:36.513098 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.513065    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhfpm\" (UniqueName: \"kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm\") pod \"c391ecfa-a9be-416f-ae45-98d453324ca6\" (UID: \"c391ecfa-a9be-416f-ae45-98d453324ca6\") "
Sep 10 23:18:36.515321 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.515274    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm" (OuterVolumeSpecName: "kube-api-access-vhfpm") pod "c391ecfa-a9be-416f-ae45-98d453324ca6" (UID: "c391ecfa-a9be-416f-ae45-98d453324ca6"). InnerVolumeSpecName "kube-api-access-vhfpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:18:36.614886 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:36.614843    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhfpm\" (UniqueName: \"kubernetes.io/projected/c391ecfa-a9be-416f-ae45-98d453324ca6-kube-api-access-vhfpm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:18:37.285727 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:37.285689    2565 generic.go:356] "Generic (PLEG): container finished" podID="c391ecfa-a9be-416f-ae45-98d453324ca6" containerID="15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6" exitCode=137
Sep 10 23:18:37.285727 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:37.285729    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:37.286181 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:37.285748    2565 scope.go:117] "RemoveContainer" containerID="15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6"
Sep 10 23:18:37.296996 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:37.296955    2565 scope.go:117] "RemoveContainer" containerID="15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6"
Sep 10 23:18:37.297070 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:37.297034    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-d3e891\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:37.297711 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:37.297688    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6\": container with ID starting with 15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6 not found: ID does not exist" containerID="15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6"
Sep 10 23:18:37.297809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:37.297718    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6"} err="failed to get container status \"15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6\": rpc error: code = NotFound desc = could not find container \"15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6\": container with ID starting with 15d4e435735efaa14db0ff014dcd90f3dc16dbad475ceae6bd8c64ded15c3be6 not found: ID does not exist"
Sep 10 23:18:38.207394 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:38.207347    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-d3e891\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" pod="opendatahub/test-curl-7655-d3e891"
Sep 10 23:18:38.209990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:38.209968    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c391ecfa-a9be-416f-ae45-98d453324ca6" path="/var/lib/kubelet/pods/c391ecfa-a9be-416f-ae45-98d453324ca6/volumes"
Sep 10 23:18:38.456535 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:38.456511    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:38.532320 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:38.532226    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b86nx\" (UniqueName: \"kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx\") pod \"d10fcfcc-ed6f-47cc-b562-6279f6769eb3\" (UID: \"d10fcfcc-ed6f-47cc-b562-6279f6769eb3\") "
Sep 10 23:18:38.534776 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:38.534744    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx" (OuterVolumeSpecName: "kube-api-access-b86nx") pod "d10fcfcc-ed6f-47cc-b562-6279f6769eb3" (UID: "d10fcfcc-ed6f-47cc-b562-6279f6769eb3"). InnerVolumeSpecName "kube-api-access-b86nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:18:38.633410 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:38.633372    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b86nx\" (UniqueName: \"kubernetes.io/projected/d10fcfcc-ed6f-47cc-b562-6279f6769eb3-kube-api-access-b86nx\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:18:39.299378 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.299342    2565 generic.go:356] "Generic (PLEG): container finished" podID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" containerID="50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed" exitCode=137
Sep 10 23:18:39.299570 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.299383    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:39.299570 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.299433    2565 scope.go:117] "RemoveContainer" containerID="50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed"
Sep 10 23:18:39.308273 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.308252    2565 scope.go:117] "RemoveContainer" containerID="50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed"
Sep 10 23:18:39.308553 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:39.308533    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed\": container with ID starting with 50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed not found: ID does not exist" containerID="50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed"
Sep 10 23:18:39.308597 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.308559    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed"} err="failed to get container status \"50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed\": rpc error: code = NotFound desc = could not find container \"50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed\": container with ID starting with 50e038f940f8256e7f1abc3c08061818b74ff0ba148c352b4a5b94a13fa626ed not found: ID does not exist"
Sep 10 23:18:39.308640 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:39.308586    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-e71835\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:39.578981 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.578894    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:18:39.579349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.579307    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" containerName="test-curl-7655-e71835"
Sep 10 23:18:39.579349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.579319    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" containerName="test-curl-7655-e71835"
Sep 10 23:18:39.579427 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.579396    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" containerName="test-curl-7655-e71835"
Sep 10 23:18:39.583923 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.583903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.585642 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:18:39.585614    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-e71835\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" pod="opendatahub/test-curl-7655-e71835"
Sep 10 23:18:39.586100 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.586075    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 23:18:39.586468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.586437    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6d5278a1-serving-cert\""
Sep 10 23:18:39.586535 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.586474    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:18:39.586756 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.586738    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 23:18:39.586852 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.586763    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9dl9w\""
Sep 10 23:18:39.593189 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.593170    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:18:39.642544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.642514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.642693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.642573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.642693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.642632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szjms\" (UniqueName: \"kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.642801 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.642717    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.744209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.744175    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.744391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.744246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.744391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.744325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.744391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.744357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szjms\" (UniqueName: \"kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.745091 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.745064    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.746807 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.746786    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.746925 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.746907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.751472 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.751451    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szjms\" (UniqueName: \"kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms\") pod \"maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:39.895727 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:39.895642    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:40.176999 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:40.176967    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:18:40.179330 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:18:40.179303    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a7c67e_6c0f_4376_819b_bf555d9ebfc5.slice/crio-b70c6f85ae31135d963a0c5da00edf738601017c87072db5c5e02d70a24f0251 WatchSource:0}: Error finding container b70c6f85ae31135d963a0c5da00edf738601017c87072db5c5e02d70a24f0251: Status 404 returned error can't find the container with id b70c6f85ae31135d963a0c5da00edf738601017c87072db5c5e02d70a24f0251
Sep 10 23:18:40.210299 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:40.210266    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d10fcfcc-ed6f-47cc-b562-6279f6769eb3" path="/var/lib/kubelet/pods/d10fcfcc-ed6f-47cc-b562-6279f6769eb3/volumes"
Sep 10 23:18:40.306141 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:40.306100    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" event={"ID":"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5","Type":"ContainerStarted","Data":"b70c6f85ae31135d963a0c5da00edf738601017c87072db5c5e02d70a24f0251"}
Sep 10 23:18:43.328131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:43.328092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" event={"ID":"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5","Type":"ContainerStarted","Data":"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"}
Sep 10 23:18:43.330571 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:43.328219    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:43.349037 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:43.348977    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" podStartSLOduration=2.055858611 podStartE2EDuration="4.348959027s" totalImagesPullingTime="2.293100416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:18:43.346651076 +0000 UTC m=+1245.633068037" watchObservedRunningTime="2026-09-10 23:18:43.348959027 +0000 UTC m=+1245.635375973"
Sep 10 23:18:45.251210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:45.251175    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:18:49.338504 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:49.338413    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:18:58.302711 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:18:58.302679    2565 scope.go:117] "RemoveContainer" containerID="5ea6f9bbd58137380fb7e1a83fb54f24d934916c56259723533c7b7acef20e80"
Sep 10 23:19:22.656640 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.656607    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:19:22.663591 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.663568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.666412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.666381    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2da572\"/\"openshift-service-ca.crt\""
Sep 10 23:19:22.666695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.666676    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2da572\"/\"default-dockercfg-cc4n9\""
Sep 10 23:19:22.666957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.666921    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-2da572\"/\"kube-root-ca.crt\""
Sep 10 23:19:22.667047 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.667023    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2da572\"/\"auth-test-model-2da572-kserve-self-signed-certs\""
Sep 10 23:19:22.668666 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.668621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:19:22.761848 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.761814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.761848 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.761853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzpgm\" (UniqueName: \"kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.762079 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.761876    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.762079 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.761970    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.762079 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.762012    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.762079 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.762048    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.862646 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.862836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.862836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862768    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.862836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzpgm\" (UniqueName: \"kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.862836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862831    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.863075 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.862891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.863219 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.863192    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.863381 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.863318    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.863381 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.863355    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.865421 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.865399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.865514 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.865498    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.871884 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.871863    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzpgm\" (UniqueName: \"kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm\") pod \"auth-test-model-2da572-kserve-56b7ff9747-l95jf\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") " pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:22.975270 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:22.975226    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:23.247143 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:23.247119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:19:23.249411 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:19:23.249377    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4917513f_f572_4eb5_aa07_c02024998238.slice/crio-7b3c5c0882973c402b08658481eac6e3cb2c09c0e45e94b8cfe47f498ed02a82 WatchSource:0}: Error finding container 7b3c5c0882973c402b08658481eac6e3cb2c09c0e45e94b8cfe47f498ed02a82: Status 404 returned error can't find the container with id 7b3c5c0882973c402b08658481eac6e3cb2c09c0e45e94b8cfe47f498ed02a82
Sep 10 23:19:23.541428 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:23.541336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" event={"ID":"4917513f-f572-4eb5-aa07-c02024998238","Type":"ContainerStarted","Data":"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"}
Sep 10 23:19:23.541428 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:23.541377    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" event={"ID":"4917513f-f572-4eb5-aa07-c02024998238","Type":"ContainerStarted","Data":"7b3c5c0882973c402b08658481eac6e3cb2c09c0e45e94b8cfe47f498ed02a82"}
Sep 10 23:19:23.557910 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:23.557862    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" podStartSLOduration=1.5578464589999999 podStartE2EDuration="1.557846459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:19:23.55763987 +0000 UTC m=+1285.844056817" watchObservedRunningTime="2026-09-10 23:19:23.557846459 +0000 UTC m=+1285.844263406"
Sep 10 23:19:26.956311 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:26.956264    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:19:32.975467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:32.975422    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:32.975467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:32.975473    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:32.988108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:32.988083    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:33.604208 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.604180    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:19:33.964438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.964401    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:19:33.967056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.967037    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:33.969348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.969328    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6d5278a1\"/\"openshift-service-ca.crt\""
Sep 10 23:19:33.969450 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.969346    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6d5278a1\"/\"kube-root-ca.crt\""
Sep 10 23:19:33.969450 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.969329    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6d5278a1\"/\"default-dockercfg-zggbl\""
Sep 10 23:19:33.969450 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.969334    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6d5278a1\"/\"e2e-lifecycle-model-6d5278a1-kserve-self-signed-certs\""
Sep 10 23:19:33.976618 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:33.976595    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:19:34.074456 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074415    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkb2p\" (UniqueName: \"kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.074634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074484    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.074634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.074634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.074634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074611    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.074777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.074663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.175910 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.175870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkb2p\" (UniqueName: \"kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.175925    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176160 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176075    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176160 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176258 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176345 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176334    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176440 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176420    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.176567 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.176548    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.178643 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.178609    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.178838 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.178822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.184377 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.184360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkb2p\" (UniqueName: \"kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p\") pod \"e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") " pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.277788 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.277704    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:34.548449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.548424    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:19:34.550909 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:19:34.550883    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04753398_c66c_49ca_acae_de49b7fd87b3.slice/crio-5bfbbd8fcee8d7f27210971afa7b8f2e50173aaf85ba243c76c20276007e3237 WatchSource:0}: Error finding container 5bfbbd8fcee8d7f27210971afa7b8f2e50173aaf85ba243c76c20276007e3237: Status 404 returned error can't find the container with id 5bfbbd8fcee8d7f27210971afa7b8f2e50173aaf85ba243c76c20276007e3237
Sep 10 23:19:34.600533 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:34.600503    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" event={"ID":"04753398-c66c-49ca-acae-de49b7fd87b3","Type":"ContainerStarted","Data":"5bfbbd8fcee8d7f27210971afa7b8f2e50173aaf85ba243c76c20276007e3237"}
Sep 10 23:19:35.607489 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:35.607454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" event={"ID":"04753398-c66c-49ca-acae-de49b7fd87b3","Type":"ContainerStarted","Data":"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"}
Sep 10 23:19:35.624674 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:35.624617    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" podStartSLOduration=2.624599514 podStartE2EDuration="2.624599514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:19:35.622949269 +0000 UTC m=+1297.909366218" watchObservedRunningTime="2026-09-10 23:19:35.624599514 +0000 UTC m=+1297.911016460"
Sep 10 23:19:39.058721 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:39.058684    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:19:44.278654 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:44.278616    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:44.279039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:44.278668    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:44.291132 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:44.291091    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:44.665849 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:44.665773    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:19:58.334703 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:58.334676    2565 scope.go:117] "RemoveContainer" containerID="b7a03a11ab31e693f6f5f2ec7659f4f474d28fa78997c6b877eb8dc31a193f09"
Sep 10 23:19:58.343483 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:19:58.343461    2565 scope.go:117] "RemoveContainer" containerID="5fda273dad65a5ca1f904dd87005205b74572be97076dbc6ae4c9a621cf4baad"
Sep 10 23:20:14.451782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:14.451749    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:20:23.551931 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.551843    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:20:23.556936 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.556910    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.559164 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.559140    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"default-dockercfg-pfgst\""
Sep 10 23:20:23.559279 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.559142    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"kube-root-ca.crt\""
Sep 10 23:20:23.560157 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.560138    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"auth-test-model-4b268f-kserve-self-signed-certs\""
Sep 10 23:20:23.560260 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.560171    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"openshift-service-ca.crt\""
Sep 10 23:20:23.564205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.564121    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:20:23.662977 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.662942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftn56\" (UniqueName: \"kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.662977 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.662978    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.663209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.663083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.663209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.663121    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.663209 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.663149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.663363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.663213    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764356 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764318    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764382    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftn56\" (UniqueName: \"kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764447    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764697 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764697 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764804 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764781    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764844 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.764901 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.764884    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.767051 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.767023    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.767207 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.767187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.772706 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.772680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftn56\" (UniqueName: \"kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56\") pod \"auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:23.868417 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:23.868333    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:24.144874 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:24.144844    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:20:24.147145 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:20:24.147113    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4bb7727_c8eb_4c8b_96fb_4b09a144e31b.slice/crio-1665b9b142352b43f4b2294052692e9b729a727e6a142d761f7a2a14464fa83d WatchSource:0}: Error finding container 1665b9b142352b43f4b2294052692e9b729a727e6a142d761f7a2a14464fa83d: Status 404 returned error can't find the container with id 1665b9b142352b43f4b2294052692e9b729a727e6a142d761f7a2a14464fa83d
Sep 10 23:20:24.858811 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:24.858777    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" event={"ID":"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b","Type":"ContainerStarted","Data":"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"}
Sep 10 23:20:24.858811 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:24.858818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" event={"ID":"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b","Type":"ContainerStarted","Data":"1665b9b142352b43f4b2294052692e9b729a727e6a142d761f7a2a14464fa83d"}
Sep 10 23:20:24.875956 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:24.875905    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" podStartSLOduration=1.8758883549999998 podStartE2EDuration="1.875888355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:20:24.874629764 +0000 UTC m=+1347.161046711" watchObservedRunningTime="2026-09-10 23:20:24.875888355 +0000 UTC m=+1347.162305302"
Sep 10 23:20:27.657624 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:27.657587    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:20:33.868551 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:33.868504    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:33.869062 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:33.868577    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:33.881139 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:33.881109    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:33.915777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:33.915747    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:20:34.175930 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.175903    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:20:34.176138 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.176117    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerName="maas-api" containerID="cri-o://09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01" gracePeriod=30
Sep 10 23:20:34.333390 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.333351    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.50:8443/health\": dial tcp 10.132.0.50:8443: connect: connection refused"
Sep 10 23:20:34.436394 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.436328    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:20:34.582039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.581999    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls\") pod \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") "
Sep 10 23:20:34.582211 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.582110    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs\") pod \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") "
Sep 10 23:20:34.582211 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.582155    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szjms\" (UniqueName: \"kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms\") pod \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") "
Sep 10 23:20:34.582348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.582214    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca\") pod \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\" (UID: \"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5\") "
Sep 10 23:20:34.582705 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.582672    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" (UID: "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:20:34.584595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.584565    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" (UID: "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:20:34.584595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.584578    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms" (OuterVolumeSpecName: "kube-api-access-szjms") pod "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" (UID: "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5"). InnerVolumeSpecName "kube-api-access-szjms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:20:34.584595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.584584    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" (UID: "e9a7c67e-6c0f-4376-819b-bf555d9ebfc5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:20:34.683993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.683950    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-openshift-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:34.683993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.683985    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-maas-api-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:34.683993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.683997    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-metrics-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:34.684234 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.684010    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szjms\" (UniqueName: \"kubernetes.io/projected/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5-kube-api-access-szjms\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:34.910963 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.910876    2565 generic.go:356] "Generic (PLEG): container finished" podID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerID="09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01" exitCode=0
Sep 10 23:20:34.910963 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.910927    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"
Sep 10 23:20:34.911553 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.910966    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" event={"ID":"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5","Type":"ContainerDied","Data":"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"}
Sep 10 23:20:34.911553 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.911019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9" event={"ID":"e9a7c67e-6c0f-4376-819b-bf555d9ebfc5","Type":"ContainerDied","Data":"b70c6f85ae31135d963a0c5da00edf738601017c87072db5c5e02d70a24f0251"}
Sep 10 23:20:34.911553 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.911047    2565 scope.go:117] "RemoveContainer" containerID="09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"
Sep 10 23:20:34.920855 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.920837    2565 scope.go:117] "RemoveContainer" containerID="09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"
Sep 10 23:20:34.921151 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:20:34.921134    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01\": container with ID starting with 09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01 not found: ID does not exist" containerID="09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"
Sep 10 23:20:34.921203 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.921160    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01"} err="failed to get container status \"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01\": rpc error: code = NotFound desc = could not find container \"09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01\": container with ID starting with 09936934f87be458f912ecc23b17cec338d9fa193253a011b978aae53b025b01 not found: ID does not exist"
Sep 10 23:20:34.931867 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.931843    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:20:34.937415 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:34.937394    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6d5278a1-577c6b9d47-vg9v9"]
Sep 10 23:20:36.209997 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:36.209960    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" path="/var/lib/kubelet/pods/e9a7c67e-6c0f-4376-819b-bf555d9ebfc5/volumes"
Sep 10 23:20:42.523246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:42.523212    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:20:47.652183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:47.652144    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:20:57.543579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.543532    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" podUID="04753398-c66c-49ca-acae-de49b7fd87b3" containerName="main" containerID="cri-o://3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa" gracePeriod=45
Sep 10 23:20:57.797238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.797172    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:20:57.814529 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814498    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814721 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814556    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814721 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814637    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkb2p\" (UniqueName: \"kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814721 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814664    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814860 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814739    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814860 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814785    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache\") pod \"04753398-c66c-49ca-acae-de49b7fd87b3\" (UID: \"04753398-c66c-49ca-acae-de49b7fd87b3\") "
Sep 10 23:20:57.814860 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.814797    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:20:57.815026 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.815005    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home" (OuterVolumeSpecName: "home") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:20:57.815148 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.815125    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:57.815323 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.815281    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:57.815323 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.815243    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:20:57.817488 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.817441    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm" (OuterVolumeSpecName: "dshm") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:20:57.817605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.817493    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:20:57.817605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.817494    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p" (OuterVolumeSpecName: "kube-api-access-pkb2p") pod "04753398-c66c-49ca-acae-de49b7fd87b3" (UID: "04753398-c66c-49ca-acae-de49b7fd87b3"). InnerVolumeSpecName "kube-api-access-pkb2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:20:57.916324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.916277    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04753398-c66c-49ca-acae-de49b7fd87b3-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:57.916324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.916322    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkb2p\" (UniqueName: \"kubernetes.io/projected/04753398-c66c-49ca-acae-de49b7fd87b3-kube-api-access-pkb2p\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:57.916324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.916331    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:57.916568 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:57.916341    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04753398-c66c-49ca-acae-de49b7fd87b3-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:20:58.040635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.040602    2565 generic.go:356] "Generic (PLEG): container finished" podID="04753398-c66c-49ca-acae-de49b7fd87b3" containerID="3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa" exitCode=0
Sep 10 23:20:58.040833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.040649    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"
Sep 10 23:20:58.040833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.040695    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" event={"ID":"04753398-c66c-49ca-acae-de49b7fd87b3","Type":"ContainerDied","Data":"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"}
Sep 10 23:20:58.040833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.040739    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr" event={"ID":"04753398-c66c-49ca-acae-de49b7fd87b3","Type":"ContainerDied","Data":"5bfbbd8fcee8d7f27210971afa7b8f2e50173aaf85ba243c76c20276007e3237"}
Sep 10 23:20:58.040833 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.040763    2565 scope.go:117] "RemoveContainer" containerID="3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"
Sep 10 23:20:58.050451 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.050432    2565 scope.go:117] "RemoveContainer" containerID="3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"
Sep 10 23:20:58.050782 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:20:58.050760    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa\": container with ID starting with 3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa not found: ID does not exist" containerID="3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"
Sep 10 23:20:58.050825 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.050794    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa"} err="failed to get container status \"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa\": rpc error: code = NotFound desc = could not find container \"3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa\": container with ID starting with 3617fa89259e785567bb31b409b79a2011dae51ad5ce49679b79b8dbb9d85daa not found: ID does not exist"
Sep 10 23:20:58.061569 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.061543    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:20:58.067348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.067318    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6d5278a1/e2e-lifecycle-model-6d5278a1-kserve-84d78f5f87-8f9zr"]
Sep 10 23:20:58.210946 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.210904    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04753398-c66c-49ca-acae-de49b7fd87b3" path="/var/lib/kubelet/pods/04753398-c66c-49ca-acae-de49b7fd87b3/volumes"
Sep 10 23:20:58.368744 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:20:58.368659    2565 scope.go:117] "RemoveContainer" containerID="1c5acca67e40e4e4128078eb036a7dace27845ba2fd7e8b53ae55d62a274b006"
Sep 10 23:21:00.761504 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:21:00.761468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:21:16.558246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:21:16.558205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:21:32.452853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:21:32.452823    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:21:47.192443 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:21:47.192411    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:21:58.396361 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:21:58.396258    2565 scope.go:117] "RemoveContainer" containerID="1d32cedf5cab6c821c81894865fec1e360cda3c85d11c7703bce86e58e081df2"
Sep 10 23:22:03.151415 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:03.151356    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:22:13.386810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.386769    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7658-22d67f"]
Sep 10 23:22:13.387433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387368    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04753398-c66c-49ca-acae-de49b7fd87b3" containerName="main"
Sep 10 23:22:13.387433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387391    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04753398-c66c-49ca-acae-de49b7fd87b3" containerName="main"
Sep 10 23:22:13.387433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387406    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerName="maas-api"
Sep 10 23:22:13.387433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387415    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerName="maas-api"
Sep 10 23:22:13.387662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387525    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a7c67e-6c0f-4376-819b-bf555d9ebfc5" containerName="maas-api"
Sep 10 23:22:13.387662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.387541    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04753398-c66c-49ca-acae-de49b7fd87b3" containerName="main"
Sep 10 23:22:13.391965 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.391944    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:13.394517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.394492    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nn896\""
Sep 10 23:22:13.396348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.396327    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-22d67f"]
Sep 10 23:22:13.462919 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.462879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9m26\" (UniqueName: \"kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26\") pod \"test-curl-7658-22d67f\" (UID: \"2eef20f9-b395-43f4-be11-2c2714821368\") " pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:13.564112 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.564068    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9m26\" (UniqueName: \"kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26\") pod \"test-curl-7658-22d67f\" (UID: \"2eef20f9-b395-43f4-be11-2c2714821368\") " pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:13.572876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.572842    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9m26\" (UniqueName: \"kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26\") pod \"test-curl-7658-22d67f\" (UID: \"2eef20f9-b395-43f4-be11-2c2714821368\") " pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:13.702748 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.702706    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:13.973831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:13.973809    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-22d67f"]
Sep 10 23:22:13.976133 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:22:13.976100    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eef20f9_b395_43f4_be11_2c2714821368.slice/crio-ab284922d487fca411c267aed8a99fc5e22b00e8f99ed6d25254706f8c241e13 WatchSource:0}: Error finding container ab284922d487fca411c267aed8a99fc5e22b00e8f99ed6d25254706f8c241e13: Status 404 returned error can't find the container with id ab284922d487fca411c267aed8a99fc5e22b00e8f99ed6d25254706f8c241e13
Sep 10 23:22:14.436305 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:14.436255    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-22d67f" event={"ID":"2eef20f9-b395-43f4-be11-2c2714821368","Type":"ContainerStarted","Data":"660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9"}
Sep 10 23:22:14.436703 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:14.436331    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-22d67f" event={"ID":"2eef20f9-b395-43f4-be11-2c2714821368","Type":"ContainerStarted","Data":"ab284922d487fca411c267aed8a99fc5e22b00e8f99ed6d25254706f8c241e13"}
Sep 10 23:22:14.449762 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:14.449698    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7658-22d67f" podStartSLOduration=1.449678703 podStartE2EDuration="1.449678703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:22:14.449347938 +0000 UTC m=+1456.735764886" watchObservedRunningTime="2026-09-10 23:22:14.449678703 +0000 UTC m=+1456.736095653"
Sep 10 23:22:15.047761 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:15.047720    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7658-22d67f"]
Sep 10 23:22:15.051693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:15.051666    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7658-22d67f"]
Sep 10 23:22:15.441649 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:22:15.441617    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7658-22d67f" container="test-curl-7658-22d67f"
Sep 10 23:22:16.396431 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:16.396400    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:22:16.447097 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:16.447057    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7658-22d67f" podUID="2eef20f9-b395-43f4-be11-2c2714821368" containerName="test-curl-7658-22d67f" containerID="cri-o://660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9" gracePeriod=2
Sep 10 23:22:17.477117 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:17.477076    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:22:18.622798 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:18.622773    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:18.714411 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:18.714330    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9m26\" (UniqueName: \"kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26\") pod \"2eef20f9-b395-43f4-be11-2c2714821368\" (UID: \"2eef20f9-b395-43f4-be11-2c2714821368\") "
Sep 10 23:22:18.716623 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:18.716586    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26" (OuterVolumeSpecName: "kube-api-access-v9m26") pod "2eef20f9-b395-43f4-be11-2c2714821368" (UID: "2eef20f9-b395-43f4-be11-2c2714821368"). InnerVolumeSpecName "kube-api-access-v9m26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:22:18.747832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:18.747799    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:22:18.815596 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:18.815554    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9m26\" (UniqueName: \"kubernetes.io/projected/2eef20f9-b395-43f4-be11-2c2714821368-kube-api-access-v9m26\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:19.464634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:19.464596    2565 generic.go:356] "Generic (PLEG): container finished" podID="2eef20f9-b395-43f4-be11-2c2714821368" containerID="660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9" exitCode=137
Sep 10 23:22:19.464634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:19.464641    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:19.464853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:19.464661    2565 scope.go:117] "RemoveContainer" containerID="660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9"
Sep 10 23:22:19.473997 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:19.473980    2565 scope.go:117] "RemoveContainer" containerID="660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9"
Sep 10 23:22:19.474274 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:22:19.474258    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9\": container with ID starting with 660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9 not found: ID does not exist" containerID="660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9"
Sep 10 23:22:19.474350 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:19.474281    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9"} err="failed to get container status \"660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9\": rpc error: code = NotFound desc = could not find container \"660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9\": container with ID starting with 660c4333b4d933a9a009a3adfaed49ec2baa409b608a25df06643b604b42e0f9 not found: ID does not exist"
Sep 10 23:22:19.474789 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:22:19.474763    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7658-22d67f\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="2eef20f9-b395-43f4-be11-2c2714821368" pod="opendatahub/test-curl-7658-22d67f"
Sep 10 23:22:20.210518 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:20.210485    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eef20f9-b395-43f4-be11-2c2714821368" path="/var/lib/kubelet/pods/2eef20f9-b395-43f4-be11-2c2714821368/volumes"
Sep 10 23:22:22.951462 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.951425    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:22:22.951873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.951811    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eef20f9-b395-43f4-be11-2c2714821368" containerName="test-curl-7658-22d67f"
Sep 10 23:22:22.951873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.951823    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eef20f9-b395-43f4-be11-2c2714821368" containerName="test-curl-7658-22d67f"
Sep 10 23:22:22.951957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.951903    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eef20f9-b395-43f4-be11-2c2714821368" containerName="test-curl-7658-22d67f"
Sep 10 23:22:22.956464 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.956444    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:22.958435 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.958407    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-2da572\"/\"sub-test-model-2da572-kserve-self-signed-certs\""
Sep 10 23:22:22.966543 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:22.966515    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:22:23.053498 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.053498 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.053736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053584    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.053736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztx9k\" (UniqueName: \"kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.053736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.053736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.053686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.154771 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154725    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.154771 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155046 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155046 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154859    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155046 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155046 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.154936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztx9k\" (UniqueName: \"kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155283 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.155260    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155366 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.155334    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.155411 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.155366    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.157238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.157206    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.157630 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.157607    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.164082 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.164061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztx9k\" (UniqueName: \"kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k\") pod \"sub-test-model-2da572-kserve-6d6c54d85-cv82c\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") " pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.268538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.268447    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:23.544616 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:23.544590    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:22:23.546955 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:22:23.546926    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd789d6c8_c21a_468a_9695_c9a28a4d6172.slice/crio-b32ee84693d1c380211af1db22c5e26a49d6fc8c6c581f3f33340c784c01f2ce WatchSource:0}: Error finding container b32ee84693d1c380211af1db22c5e26a49d6fc8c6c581f3f33340c784c01f2ce: Status 404 returned error can't find the container with id b32ee84693d1c380211af1db22c5e26a49d6fc8c6c581f3f33340c784c01f2ce
Sep 10 23:22:24.495133 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:24.495094    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" event={"ID":"d789d6c8-c21a-468a-9695-c9a28a4d6172","Type":"ContainerStarted","Data":"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"}
Sep 10 23:22:24.495133 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:24.495145    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" event={"ID":"d789d6c8-c21a-468a-9695-c9a28a4d6172","Type":"ContainerStarted","Data":"b32ee84693d1c380211af1db22c5e26a49d6fc8c6c581f3f33340c784c01f2ce"}
Sep 10 23:22:24.514021 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:24.513970    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" podStartSLOduration=2.513954019 podStartE2EDuration="2.513954019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:22:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:22:24.512032041 +0000 UTC m=+1466.798449025" watchObservedRunningTime="2026-09-10 23:22:24.513954019 +0000 UTC m=+1466.800370996"
Sep 10 23:22:31.078957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.078920    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:22:31.082508 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.082489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.084375 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.084351    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-5052bd-serving-cert\""
Sep 10 23:22:31.084612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.084599    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 23:22:31.084665 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.084628    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:22:31.085320 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.085274    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9dl9w\""
Sep 10 23:22:31.085410 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.085362    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 23:22:31.091924 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.091867    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:22:31.237388 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.237350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.237578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.237421    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qhc7\" (UniqueName: \"kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.237578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.237508    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.237578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.237523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.339019 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.338910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.339019 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.338961    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.339019 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.339006    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.339353 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.339178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qhc7\" (UniqueName: \"kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.339674 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.339650    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.341708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.341684    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.341803 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.341736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.347831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.347808    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qhc7\" (UniqueName: \"kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7\") pod \"maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.395569 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.395536    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:31.416792 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.416745    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" podUID="4917513f-f572-4eb5-aa07-c02024998238" containerName="main" containerID="cri-o://a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d" gracePeriod=45
Sep 10 23:22:31.667372 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.667335    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:22:31.671096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.671066    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:22:31.673751 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:22:31.673727    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a6dcc0_0635_4dfa_bd4d_a3feb5257e00.slice/crio-c4b8e5ad5fb3bd412414a6b4ccba75359eee6a36c44269a736e5a57eacad692e WatchSource:0}: Error finding container c4b8e5ad5fb3bd412414a6b4ccba75359eee6a36c44269a736e5a57eacad692e: Status 404 returned error can't find the container with id c4b8e5ad5fb3bd412414a6b4ccba75359eee6a36c44269a736e5a57eacad692e
Sep 10 23:22:31.742627 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742596    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.742821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742649    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.742821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742715    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzpgm\" (UniqueName: \"kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.742821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742750    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.742821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742805    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.743034 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742905    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:31.743100 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.743068    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache" (OuterVolumeSpecName: "model-cache") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:31.743196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.742828    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home\") pod \"4917513f-f572-4eb5-aa07-c02024998238\" (UID: \"4917513f-f572-4eb5-aa07-c02024998238\") "
Sep 10 23:22:31.743260 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.743189    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home" (OuterVolumeSpecName: "home") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:31.743546 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.743528    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:31.743645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.743548    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:31.743645 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.743559    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:31.745231 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.745206    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm" (OuterVolumeSpecName: "kube-api-access-xzpgm") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "kube-api-access-xzpgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:22:31.745359 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.745339    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:22:31.745414 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.745366    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm" (OuterVolumeSpecName: "dshm") pod "4917513f-f572-4eb5-aa07-c02024998238" (UID: "4917513f-f572-4eb5-aa07-c02024998238"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:31.844825 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.844786    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4917513f-f572-4eb5-aa07-c02024998238-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:31.844825 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.844815    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzpgm\" (UniqueName: \"kubernetes.io/projected/4917513f-f572-4eb5-aa07-c02024998238-kube-api-access-xzpgm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:31.844825 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:31.844824    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4917513f-f572-4eb5-aa07-c02024998238-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.500008 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.499963    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" podUID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" containerName="main" containerID="cri-o://bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818" gracePeriod=45
Sep 10 23:22:32.537306 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.537257    2565 generic.go:356] "Generic (PLEG): container finished" podID="4917513f-f572-4eb5-aa07-c02024998238" containerID="a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d" exitCode=0
Sep 10 23:22:32.537467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.537327    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"
Sep 10 23:22:32.537467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.537329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" event={"ID":"4917513f-f572-4eb5-aa07-c02024998238","Type":"ContainerDied","Data":"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"}
Sep 10 23:22:32.537467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.537373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf" event={"ID":"4917513f-f572-4eb5-aa07-c02024998238","Type":"ContainerDied","Data":"7b3c5c0882973c402b08658481eac6e3cb2c09c0e45e94b8cfe47f498ed02a82"}
Sep 10 23:22:32.537467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.537396    2565 scope.go:117] "RemoveContainer" containerID="a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"
Sep 10 23:22:32.538967 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.538948    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" event={"ID":"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00","Type":"ContainerStarted","Data":"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"}
Sep 10 23:22:32.539085 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.539016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" event={"ID":"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00","Type":"ContainerStarted","Data":"c4b8e5ad5fb3bd412414a6b4ccba75359eee6a36c44269a736e5a57eacad692e"}
Sep 10 23:22:32.539153 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.539082    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:32.559734 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.559678    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" podStartSLOduration=1.197341582 podStartE2EDuration="1.559660968s" totalImagesPullingTime="362.319386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:22:32.558054621 +0000 UTC m=+1474.844471565" watchObservedRunningTime="2026-09-10 23:22:32.559660968 +0000 UTC m=+1474.846077915"
Sep 10 23:22:32.572928 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.572895    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:22:32.576531 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.576501    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/auth-test-model-2da572-kserve-56b7ff9747-l95jf"]
Sep 10 23:22:32.612447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.612426    2565 scope.go:117] "RemoveContainer" containerID="a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"
Sep 10 23:22:32.612795 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:22:32.612773    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d\": container with ID starting with a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d not found: ID does not exist" containerID="a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"
Sep 10 23:22:32.612891 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.612803    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d"} err="failed to get container status \"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d\": rpc error: code = NotFound desc = could not find container \"a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d\": container with ID starting with a612c9853c9a53c54b0ab28f6f96832464217cb774afb84c017a4482f540a41d not found: ID does not exist"
Sep 10 23:22:32.757035 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.757004    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:22:32.853391 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853357    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853419    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853451    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853487    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftn56\" (UniqueName: \"kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853513    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853571    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home\") pod \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\" (UID: \"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b\") "
Sep 10 23:22:32.853857 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853686    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:32.853912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853857    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:32.853912 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853886    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home" (OuterVolumeSpecName: "home") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:32.853999 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.853981    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.854056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.854002    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.854056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.854011    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.856030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.856009    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56" (OuterVolumeSpecName: "kube-api-access-ftn56") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "kube-api-access-ftn56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:22:32.856174 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.856158    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm" (OuterVolumeSpecName: "dshm") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:22:32.856174 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.856164    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" (UID: "a4bb7727-c8eb-4c8b-96fb-4b09a144e31b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:22:32.955108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.955073    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.955108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.955103    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:32.955108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:32.955113    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftn56\" (UniqueName: \"kubernetes.io/projected/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b-kube-api-access-ftn56\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:22:33.054336 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.054275    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:22:33.269054 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.269015    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:33.269054 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.269065    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:33.281836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.281805    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:33.546342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.546226    2565 generic.go:356] "Generic (PLEG): container finished" podID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" containerID="bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818" exitCode=0
Sep 10 23:22:33.546342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.546283    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"
Sep 10 23:22:33.546342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.546316    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" event={"ID":"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b","Type":"ContainerDied","Data":"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"}
Sep 10 23:22:33.546907 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.546354    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw" event={"ID":"a4bb7727-c8eb-4c8b-96fb-4b09a144e31b","Type":"ContainerDied","Data":"1665b9b142352b43f4b2294052692e9b729a727e6a142d761f7a2a14464fa83d"}
Sep 10 23:22:33.546907 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.546371    2565 scope.go:117] "RemoveContainer" containerID="bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"
Sep 10 23:22:33.555773 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.555753    2565 scope.go:117] "RemoveContainer" containerID="bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"
Sep 10 23:22:33.556068 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:22:33.556050    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818\": container with ID starting with bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818 not found: ID does not exist" containerID="bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"
Sep 10 23:22:33.556130 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.556077    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818"} err="failed to get container status \"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818\": rpc error: code = NotFound desc = could not find container \"bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818\": container with ID starting with bb558ff1d958df1f8333fb0b429b313106b9f36f3d7c7652a23b39b3b052e818 not found: ID does not exist"
Sep 10 23:22:33.559074 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.559055    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:22:33.569847 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.569818    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:22:33.575888 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:33.575859    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/auth-test-model-4b268f-kserve-5f5bcd895f-q4ctw"]
Sep 10 23:22:34.210017 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:34.209980    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4917513f-f572-4eb5-aa07-c02024998238" path="/var/lib/kubelet/pods/4917513f-f572-4eb5-aa07-c02024998238/volumes"
Sep 10 23:22:34.210427 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:34.210413    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" path="/var/lib/kubelet/pods/a4bb7727-c8eb-4c8b-96fb-4b09a144e31b/volumes"
Sep 10 23:22:38.552945 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:38.552914    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:22:47.285986 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:47.285948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:22:53.074320 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074266    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:22:53.074761 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074747    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4917513f-f572-4eb5-aa07-c02024998238" containerName="main"
Sep 10 23:22:53.074831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074764    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4917513f-f572-4eb5-aa07-c02024998238" containerName="main"
Sep 10 23:22:53.074831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074785    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" containerName="main"
Sep 10 23:22:53.074831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074793    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" containerName="main"
Sep 10 23:22:53.074965 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074857    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4917513f-f572-4eb5-aa07-c02024998238" containerName="main"
Sep 10 23:22:53.074965 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.074875    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4bb7727-c8eb-4c8b-96fb-4b09a144e31b" containerName="main"
Sep 10 23:22:53.079891 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.079869    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.082600 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.082573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"sub-test-model-4b268f-kserve-self-signed-certs\""
Sep 10 23:22:53.082748 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.082629    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"default-dockercfg-pfgst\""
Sep 10 23:22:53.082827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.082762    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"kube-root-ca.crt\""
Sep 10 23:22:53.083339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.083318    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"openshift-service-ca.crt\""
Sep 10 23:22:53.085547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.085527    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:22:53.140536 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.140708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.140708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140614    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5vbz\" (UniqueName: \"kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.140708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.140708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.140708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.140696    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241204    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241242    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241492 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241492 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241492 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5vbz\" (UniqueName: \"kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241639 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241503    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241710 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241722    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.241812 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.241764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.243752 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.243728    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.243995 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.243977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.249520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.249497    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5vbz\" (UniqueName: \"kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz\") pod \"sub-test-model-4b268f-kserve-dc5f4745f-8czcr\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.391217 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.391121    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:22:53.664619 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:53.664592    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:22:53.667484 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:22:53.667449    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5973219e_e60b_4eac_a790_a85d7239210a.slice/crio-eef1699f596589ae3fbd470fd18f58fc164cc41690252619d2132ccad647342c WatchSource:0}: Error finding container eef1699f596589ae3fbd470fd18f58fc164cc41690252619d2132ccad647342c: Status 404 returned error can't find the container with id eef1699f596589ae3fbd470fd18f58fc164cc41690252619d2132ccad647342c
Sep 10 23:22:54.665764 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:54.665731    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" event={"ID":"5973219e-e60b-4eac-a790-a85d7239210a","Type":"ContainerStarted","Data":"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"}
Sep 10 23:22:54.665764 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:54.665772    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" event={"ID":"5973219e-e60b-4eac-a790-a85d7239210a","Type":"ContainerStarted","Data":"eef1699f596589ae3fbd470fd18f58fc164cc41690252619d2132ccad647342c"}
Sep 10 23:22:54.682488 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:54.682432    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" podStartSLOduration=1.682413486 podStartE2EDuration="1.682413486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:22:54.681146255 +0000 UTC m=+1496.967563213" watchObservedRunningTime="2026-09-10 23:22:54.682413486 +0000 UTC m=+1496.968830433"
Sep 10 23:22:55.684047 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.684011    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:22:55.688785 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.688760    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:22:55.688947 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.688848    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.690943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.690920    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 23:22:55.691142 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.691121    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-44c67e-openshift-default-dockercfg-nv5nv\""
Sep 10 23:22:55.766096 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766062    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766338 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766126    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766338 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766338 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd4px\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.766540 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.766485    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867343 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867279    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867486    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd4px\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867803 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867570    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867803 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867599    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867803 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867633    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867803 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867783    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.867981 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867826    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.868045 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.867991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.868095 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.868073    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.868595 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.868568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.870310 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.870271    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.870536 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.870515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.875701 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.875680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:55.875924 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:55.875904    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd4px\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px\") pod \"e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") " pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:56.001157 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:56.001054    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:22:56.280634 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:56.280607    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:22:56.283309 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:22:56.283268    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63352a3b_05f7_47ec_9d4f_9609519bbfc7.slice/crio-67531507d43e95a04db5d5bc25e054933c72c6235adeec81e34fc79803aa2c7f WatchSource:0}: Error finding container 67531507d43e95a04db5d5bc25e054933c72c6235adeec81e34fc79803aa2c7f: Status 404 returned error can't find the container with id 67531507d43e95a04db5d5bc25e054933c72c6235adeec81e34fc79803aa2c7f
Sep 10 23:22:56.677631 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:56.677599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerStarted","Data":"67531507d43e95a04db5d5bc25e054933c72c6235adeec81e34fc79803aa2c7f"}
Sep 10 23:22:59.174426 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:59.174382    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:22:59.174841 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:59.174471    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:22:59.174841 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:59.174517    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:22:59.696615 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:59.696581    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerStarted","Data":"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"}
Sep 10 23:22:59.717485 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:22:59.717438    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podStartSLOduration=1.829009905 podStartE2EDuration="4.717422471s" totalImagesPullingTime="2.888412566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:22:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:22:59.715900233 +0000 UTC m=+1502.002317180" watchObservedRunningTime="2026-09-10 23:22:59.717422471 +0000 UTC m=+1502.003839457"
Sep 10 23:23:00.002349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:00.002217    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:00.007495 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:00.007462    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:00.702743 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:00.702710    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:00.703747 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:00.703724    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:01.865254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:01.865223    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:23:02.770047 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.770011    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:23:02.776420 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.776396    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.778454 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.778426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-44c67e-serving-cert\""
Sep 10 23:23:02.785066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.785034    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:23:02.835598 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.835566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.835763 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.835624    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.835763 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.835649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.835763 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.835684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmvbn\" (UniqueName: \"kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.936219 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.936186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.936718 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.936229    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.936718 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.936250    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.936718 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.936344    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmvbn\" (UniqueName: \"kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.936881 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.936867    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.938980 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.938958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.939092 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.938985    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:02.944941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:02.944921    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmvbn\" (UniqueName: \"kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn\") pod \"maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:03.090599 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.090502    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:03.365080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.365055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:23:03.367584 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:23:03.367552    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9edc9d_71b5_4ed3_a7d4_9666034fa048.slice/crio-450baad5a3e1962563124d8b0d1e252ebbe5e600e9fc17861378c48d48923bcb WatchSource:0}: Error finding container 450baad5a3e1962563124d8b0d1e252ebbe5e600e9fc17861378c48d48923bcb: Status 404 returned error can't find the container with id 450baad5a3e1962563124d8b0d1e252ebbe5e600e9fc17861378c48d48923bcb
Sep 10 23:23:03.391536 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.391508    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:03.391798 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.391775    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:03.405493 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.405467    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:03.720316 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.720249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" event={"ID":"9e9edc9d-71b5-4ed3-a7d4-9666034fa048","Type":"ContainerStarted","Data":"450baad5a3e1962563124d8b0d1e252ebbe5e600e9fc17861378c48d48923bcb"}
Sep 10 23:23:03.731740 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:03.731707    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:04.727648 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:04.727612    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" event={"ID":"9e9edc9d-71b5-4ed3-a7d4-9666034fa048","Type":"ContainerStarted","Data":"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"}
Sep 10 23:23:04.746714 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:04.746659    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" podStartSLOduration=2.328370747 podStartE2EDuration="2.746641261s" totalImagesPullingTime="418.270514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:23:04.744345245 +0000 UTC m=+1507.030762189" watchObservedRunningTime="2026-09-10 23:23:04.746641261 +0000 UTC m=+1507.033058274"
Sep 10 23:23:05.732790 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:05.732758    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:11.743673 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:11.743637    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:23:15.958984 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:15.958945    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:23:19.811646 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.811611    2565 generic.go:356] "Generic (PLEG): container finished" podID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerID="34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6" exitCode=0
Sep 10 23:23:19.812066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.811665    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerDied","Data":"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"}
Sep 10 23:23:19.812066 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.812056    2565 scope.go:117] "RemoveContainer" containerID="34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"
Sep 10 23:23:19.814266 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.814220    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:23:19.814412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.814338    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:23:19.814412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:19.814391    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:23:20.819279 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:20.819244    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerStarted","Data":"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"}
Sep 10 23:23:21.001649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:21.001605    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:21.001954 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:21.001926    2565 patch_prober.go:28] interesting pod/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 23:23:21.002020 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:21.001997    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 23:23:22.001607 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:22.001565    2565 patch_prober.go:28] interesting pod/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 23:23:22.002023 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:22.001635    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 23:23:23.001606 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:23.001561    2565 patch_prober.go:28] interesting pod/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 23:23:23.001796 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:23.001638    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 23:23:24.001850 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:24.001812    2565 patch_prober.go:28] interesting pod/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 23:23:24.002345 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:24.001868    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 23:23:25.002183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:25.002140    2565 patch_prober.go:28] interesting pod/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 10 23:23:25.002711 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:25.002207    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 10 23:23:26.092510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:26.092475    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:26.093013 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:26.092894    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:26.093995 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:26.093972    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:23:27.434717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.431429    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7658-5ed0b6"]
Sep 10 23:23:27.436699 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.436667    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:27.438402 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.438371    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-5ed0b6"]
Sep 10 23:23:27.438932 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.438912    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nn896\""
Sep 10 23:23:27.482425 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.482399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqbq\" (UniqueName: \"kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq\") pod \"test-curl-7658-5ed0b6\" (UID: \"7c7788b8-1e9f-4859-bcbf-31194f4a6724\") " pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:27.583472 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.583437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqbq\" (UniqueName: \"kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq\") pod \"test-curl-7658-5ed0b6\" (UID: \"7c7788b8-1e9f-4859-bcbf-31194f4a6724\") " pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:27.592457 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.592427    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqbq\" (UniqueName: \"kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq\") pod \"test-curl-7658-5ed0b6\" (UID: \"7c7788b8-1e9f-4859-bcbf-31194f4a6724\") " pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:27.748016 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:27.747923    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:28.019877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:28.019673    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-5ed0b6"]
Sep 10 23:23:28.022786 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:23:28.022750    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c7788b8_1e9f_4859_bcbf_31194f4a6724.slice/crio-ab4e9bca235187951d4b4ac75d852e7287f1a0149fda70f1c930d57b10e8bc2b WatchSource:0}: Error finding container ab4e9bca235187951d4b4ac75d852e7287f1a0149fda70f1c930d57b10e8bc2b: Status 404 returned error can't find the container with id ab4e9bca235187951d4b4ac75d852e7287f1a0149fda70f1c930d57b10e8bc2b
Sep 10 23:23:28.862630 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:28.862595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-5ed0b6" event={"ID":"7c7788b8-1e9f-4859-bcbf-31194f4a6724","Type":"ContainerStarted","Data":"0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd"}
Sep 10 23:23:28.863011 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:28.862642    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-5ed0b6" event={"ID":"7c7788b8-1e9f-4859-bcbf-31194f4a6724","Type":"ContainerStarted","Data":"ab4e9bca235187951d4b4ac75d852e7287f1a0149fda70f1c930d57b10e8bc2b"}
Sep 10 23:23:28.881972 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:28.881916    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7658-5ed0b6" podStartSLOduration=1.8818986199999999 podStartE2EDuration="1.88189862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:23:28.879612331 +0000 UTC m=+1531.166029277" watchObservedRunningTime="2026-09-10 23:23:28.88189862 +0000 UTC m=+1531.168315568"
Sep 10 23:23:29.448338 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.448303    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7658-5ed0b6"]
Sep 10 23:23:29.452173 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.452149    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7658-5ed0b6"]
Sep 10 23:23:29.562429 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.562343    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7658-a751ee"]
Sep 10 23:23:29.562780 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.562767    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" containerName="test-curl-7658-5ed0b6"
Sep 10 23:23:29.562834 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.562782    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" containerName="test-curl-7658-5ed0b6"
Sep 10 23:23:29.562870 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.562852    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" containerName="test-curl-7658-5ed0b6"
Sep 10 23:23:29.565941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.565924    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:29.569601 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.569574    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-a751ee"]
Sep 10 23:23:29.601777 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.601747    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88tl\" (UniqueName: \"kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl\") pod \"test-curl-7658-a751ee\" (UID: \"14651d06-0af9-4a82-ad4b-ae9e0dc825e2\") " pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:29.702905 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.702867    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88tl\" (UniqueName: \"kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl\") pod \"test-curl-7658-a751ee\" (UID: \"14651d06-0af9-4a82-ad4b-ae9e0dc825e2\") " pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:29.711358 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.711332    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88tl\" (UniqueName: \"kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl\") pod \"test-curl-7658-a751ee\" (UID: \"14651d06-0af9-4a82-ad4b-ae9e0dc825e2\") " pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:29.868027 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:29.867946    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7658-5ed0b6" container="test-curl-7658-5ed0b6"
Sep 10 23:23:29.876695 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:29.876676    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:30.150517 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:30.150492    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7658-a751ee"]
Sep 10 23:23:30.152265 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:23:30.152236    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14651d06_0af9_4a82_ad4b_ae9e0dc825e2.slice/crio-863121e9597bf9a4da155d174046179d5f0f8139a72eeaf7535c53689da51270 WatchSource:0}: Error finding container 863121e9597bf9a4da155d174046179d5f0f8139a72eeaf7535c53689da51270: Status 404 returned error can't find the container with id 863121e9597bf9a4da155d174046179d5f0f8139a72eeaf7535c53689da51270
Sep 10 23:23:30.874542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:30.874501    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-a751ee" event={"ID":"14651d06-0af9-4a82-ad4b-ae9e0dc825e2","Type":"ContainerStarted","Data":"21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95"}
Sep 10 23:23:30.874542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:30.874524    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7658-5ed0b6" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" containerName="test-curl-7658-5ed0b6" containerID="cri-o://0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd" gracePeriod=2
Sep 10 23:23:30.874542 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:30.874547    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7658-a751ee" event={"ID":"14651d06-0af9-4a82-ad4b-ae9e0dc825e2","Type":"ContainerStarted","Data":"863121e9597bf9a4da155d174046179d5f0f8139a72eeaf7535c53689da51270"}
Sep 10 23:23:30.890210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:30.890159    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7658-a751ee" podStartSLOduration=1.890145532 podStartE2EDuration="1.890145532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:23:30.888889045 +0000 UTC m=+1533.175305988" watchObservedRunningTime="2026-09-10 23:23:30.890145532 +0000 UTC m=+1533.176562479"
Sep 10 23:23:31.473767 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:31.473730    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7658-a751ee"]
Sep 10 23:23:31.476921 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:31.476892    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7658-a751ee"]
Sep 10 23:23:31.879928 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:31.879901    2565 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7658-a751ee" container="test-curl-7658-a751ee"
Sep 10 23:23:32.885070 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:32.885037    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7658-a751ee" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" containerName="test-curl-7658-a751ee" containerID="cri-o://21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95" gracePeriod=2
Sep 10 23:23:33.063052 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.063028    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:33.135092 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.135000    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwqbq\" (UniqueName: \"kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq\") pod \"7c7788b8-1e9f-4859-bcbf-31194f4a6724\" (UID: \"7c7788b8-1e9f-4859-bcbf-31194f4a6724\") "
Sep 10 23:23:33.137235 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.137204    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq" (OuterVolumeSpecName: "kube-api-access-mwqbq") pod "7c7788b8-1e9f-4859-bcbf-31194f4a6724" (UID: "7c7788b8-1e9f-4859-bcbf-31194f4a6724"). InnerVolumeSpecName "kube-api-access-mwqbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:23:33.236382 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.236335    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwqbq\" (UniqueName: \"kubernetes.io/projected/7c7788b8-1e9f-4859-bcbf-31194f4a6724-kube-api-access-mwqbq\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:33.415677 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.415594    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:23:33.891918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.891880    2565 generic.go:356] "Generic (PLEG): container finished" podID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" containerID="0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd" exitCode=137
Sep 10 23:23:33.891918 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.891918    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:33.892366 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.891981    2565 scope.go:117] "RemoveContainer" containerID="0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd"
Sep 10 23:23:33.900996 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.900968    2565 scope.go:117] "RemoveContainer" containerID="0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd"
Sep 10 23:23:33.901253 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:33.901236    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd\": container with ID starting with 0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd not found: ID does not exist" containerID="0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd"
Sep 10 23:23:33.901330 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:33.901264    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd"} err="failed to get container status \"0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd\": rpc error: code = NotFound desc = could not find container \"0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd\": container with ID starting with 0f1c21a31e4ee16d969a9516ff111f7bbd94ac55d63318e6f6523091b3fd74cd not found: ID does not exist"
Sep 10 23:23:33.902000 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:33.901984    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7658-5ed0b6\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" pod="opendatahub/test-curl-7658-5ed0b6"
Sep 10 23:23:34.211162 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:34.211123    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c7788b8-1e9f-4859-bcbf-31194f4a6724" path="/var/lib/kubelet/pods/7c7788b8-1e9f-4859-bcbf-31194f4a6724/volumes"
Sep 10 23:23:34.433897 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:34.433860    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:23:35.055981 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.055957    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:35.153443 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.153359    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l88tl\" (UniqueName: \"kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl\") pod \"14651d06-0af9-4a82-ad4b-ae9e0dc825e2\" (UID: \"14651d06-0af9-4a82-ad4b-ae9e0dc825e2\") "
Sep 10 23:23:35.155579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.155549    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl" (OuterVolumeSpecName: "kube-api-access-l88tl") pod "14651d06-0af9-4a82-ad4b-ae9e0dc825e2" (UID: "14651d06-0af9-4a82-ad4b-ae9e0dc825e2"). InnerVolumeSpecName "kube-api-access-l88tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:23:35.254915 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.254879    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l88tl\" (UniqueName: \"kubernetes.io/projected/14651d06-0af9-4a82-ad4b-ae9e0dc825e2-kube-api-access-l88tl\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:35.905230 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.905192    2565 generic.go:356] "Generic (PLEG): container finished" podID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" containerID="21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95" exitCode=137
Sep 10 23:23:35.905432 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.905249    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:35.905432 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.905268    2565 scope.go:117] "RemoveContainer" containerID="21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95"
Sep 10 23:23:35.914655 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.914637    2565 scope.go:117] "RemoveContainer" containerID="21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95"
Sep 10 23:23:35.914967 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:35.914949    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95\": container with ID starting with 21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95 not found: ID does not exist" containerID="21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95"
Sep 10 23:23:35.915006 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:35.914979    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95"} err="failed to get container status \"21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95\": rpc error: code = NotFound desc = could not find container \"21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95\": container with ID starting with 21df8d175374ce6c01a41ef3b79de448fde105772d286b2ea4a2ef2344091f95 not found: ID does not exist"
Sep 10 23:23:35.915816 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:35.915795    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7658-a751ee\" is forbidden: User \"system:node:ip-10-0-137-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-120.ec2.internal' and this object" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" pod="opendatahub/test-curl-7658-a751ee"
Sep 10 23:23:36.210712 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:36.210682    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" path="/var/lib/kubelet/pods/14651d06-0af9-4a82-ad4b-ae9e0dc825e2/volumes"
Sep 10 23:23:36.557556 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:36.557482    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:23:45.161374 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.161339    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:23:45.161760 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.161562    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" podUID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" containerName="maas-api" containerID="cri-o://962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e" gracePeriod=30
Sep 10 23:23:45.416450 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.416390    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:23:45.558275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.558240    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs\") pod \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") "
Sep 10 23:23:45.558482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.558357    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca\") pod \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") "
Sep 10 23:23:45.558482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.558394    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qhc7\" (UniqueName: \"kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7\") pod \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") "
Sep 10 23:23:45.558482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.558422    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls\") pod \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\" (UID: \"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00\") "
Sep 10 23:23:45.558790 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.558763    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" (UID: "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:23:45.560688 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.560656    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" (UID: "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:23:45.560814 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.560722    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" (UID: "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:23:45.560814 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.560726    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7" (OuterVolumeSpecName: "kube-api-access-2qhc7") pod "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" (UID: "d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00"). InnerVolumeSpecName "kube-api-access-2qhc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:23:45.659699 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.659655    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-openshift-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:45.659699 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.659690    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qhc7\" (UniqueName: \"kubernetes.io/projected/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-kube-api-access-2qhc7\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:45.659699 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.659704    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-maas-api-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:45.659946 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.659715    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00-metrics-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:45.960262 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.960229    2565 generic.go:356] "Generic (PLEG): container finished" podID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" containerID="962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e" exitCode=0
Sep 10 23:23:45.960458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.960269    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"
Sep 10 23:23:45.960458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.960325    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" event={"ID":"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00","Type":"ContainerDied","Data":"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"}
Sep 10 23:23:45.960458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.960383    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf" event={"ID":"d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00","Type":"ContainerDied","Data":"c4b8e5ad5fb3bd412414a6b4ccba75359eee6a36c44269a736e5a57eacad692e"}
Sep 10 23:23:45.960458 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.960413    2565 scope.go:117] "RemoveContainer" containerID="962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"
Sep 10 23:23:45.969905 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.969884    2565 scope.go:117] "RemoveContainer" containerID="962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"
Sep 10 23:23:45.970159 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:45.970143    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e\": container with ID starting with 962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e not found: ID does not exist" containerID="962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"
Sep 10 23:23:45.970198 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.970168    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e"} err="failed to get container status \"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e\": rpc error: code = NotFound desc = could not find container \"962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e\": container with ID starting with 962508c02356376272f789e4e3522c99a22cd1e1afc037bb6d9ed378cc60ff7e not found: ID does not exist"
Sep 10 23:23:45.980881 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.980858    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:23:45.985737 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:45.985718    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-5052bd-5c966d9c66-wnbwf"]
Sep 10 23:23:46.211620 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:46.211528    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" path="/var/lib/kubelet/pods/d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00/volumes"
Sep 10 23:23:48.435797 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.435763    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" podUID="d789d6c8-c21a-468a-9695-c9a28a4d6172" containerName="main" containerID="cri-o://26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5" gracePeriod=45
Sep 10 23:23:48.693689 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.693629    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:23:48.798787 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798753    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.798959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798833    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztx9k\" (UniqueName: \"kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.798959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798860    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.798959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798884    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.798959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798921    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.798959 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.798951    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir\") pod \"d789d6c8-c21a-468a-9695-c9a28a4d6172\" (UID: \"d789d6c8-c21a-468a-9695-c9a28a4d6172\") "
Sep 10 23:23:48.799240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.799101    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home" (OuterVolumeSpecName: "home") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:48.799240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.799185    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache" (OuterVolumeSpecName: "model-cache") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:48.799240 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.799231    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.799392 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.799256    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:48.801205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.801181    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm" (OuterVolumeSpecName: "dshm") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:48.801205 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.801192    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:23:48.801382 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.801277    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k" (OuterVolumeSpecName: "kube-api-access-ztx9k") pod "d789d6c8-c21a-468a-9695-c9a28a4d6172" (UID: "d789d6c8-c21a-468a-9695-c9a28a4d6172"). InnerVolumeSpecName "kube-api-access-ztx9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:23:48.900088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.900060    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztx9k\" (UniqueName: \"kubernetes.io/projected/d789d6c8-c21a-468a-9695-c9a28a4d6172-kube-api-access-ztx9k\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.900088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.900085    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d789d6c8-c21a-468a-9695-c9a28a4d6172-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.900088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.900094    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.900328 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.900102    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.900328 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.900111    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d789d6c8-c21a-468a-9695-c9a28a4d6172-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:48.979092 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.979001    2565 generic.go:356] "Generic (PLEG): container finished" podID="d789d6c8-c21a-468a-9695-c9a28a4d6172" containerID="26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5" exitCode=0
Sep 10 23:23:48.979092 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.979049    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"
Sep 10 23:23:48.979265 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.979089    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" event={"ID":"d789d6c8-c21a-468a-9695-c9a28a4d6172","Type":"ContainerDied","Data":"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"}
Sep 10 23:23:48.979265 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.979137    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c" event={"ID":"d789d6c8-c21a-468a-9695-c9a28a4d6172","Type":"ContainerDied","Data":"b32ee84693d1c380211af1db22c5e26a49d6fc8c6c581f3f33340c784c01f2ce"}
Sep 10 23:23:48.979265 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.979154    2565 scope.go:117] "RemoveContainer" containerID="26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"
Sep 10 23:23:48.990477 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.989154    2565 scope.go:117] "RemoveContainer" containerID="26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"
Sep 10 23:23:48.990650 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:48.990627    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5\": container with ID starting with 26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5 not found: ID does not exist" containerID="26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"
Sep 10 23:23:48.990723 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:48.990663    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5"} err="failed to get container status \"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5\": rpc error: code = NotFound desc = could not find container \"26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5\": container with ID starting with 26ebe8a3a8c4904af23347916e62bef044e3eeae02c9c6dc7939d6f9d4acb9b5 not found: ID does not exist"
Sep 10 23:23:49.000658 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.000634    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:23:49.006571 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.006546    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-2da572/sub-test-model-2da572-kserve-6d6c54d85-cv82c"]
Sep 10 23:23:49.454063 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.454031    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" podUID="5973219e-e60b-4eac-a790-a85d7239210a" containerName="main" containerID="cri-o://eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25" gracePeriod=45
Sep 10 23:23:49.704586 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.704531    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:49.809108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809071    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809110    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809362 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809166    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809362 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809283    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5vbz\" (UniqueName: \"kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809466 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache" (OuterVolumeSpecName: "model-cache") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:49.809466 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809457    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809545 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809469    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:49.809545 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809488    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm\") pod \"5973219e-e60b-4eac-a790-a85d7239210a\" (UID: \"5973219e-e60b-4eac-a790-a85d7239210a\") "
Sep 10 23:23:49.809663 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809646    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home" (OuterVolumeSpecName: "home") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:49.809826 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809810    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.809894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809826    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.809894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.809841    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.811582 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.811556    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz" (OuterVolumeSpecName: "kube-api-access-d5vbz") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "kube-api-access-d5vbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:23:49.811582 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.811572    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:23:49.811582 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.811574    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm" (OuterVolumeSpecName: "dshm") pod "5973219e-e60b-4eac-a790-a85d7239210a" (UID: "5973219e-e60b-4eac-a790-a85d7239210a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:23:49.910785 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.910753    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5973219e-e60b-4eac-a790-a85d7239210a-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.910785 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.910782    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5973219e-e60b-4eac-a790-a85d7239210a-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.911004 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.910794    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5vbz\" (UniqueName: \"kubernetes.io/projected/5973219e-e60b-4eac-a790-a85d7239210a-kube-api-access-d5vbz\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:23:49.985001 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.984916    2565 generic.go:356] "Generic (PLEG): container finished" podID="5973219e-e60b-4eac-a790-a85d7239210a" containerID="eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25" exitCode=0
Sep 10 23:23:49.985001 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.984965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"
Sep 10 23:23:49.985275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.985004    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" event={"ID":"5973219e-e60b-4eac-a790-a85d7239210a","Type":"ContainerDied","Data":"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"}
Sep 10 23:23:49.985275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.985053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr" event={"ID":"5973219e-e60b-4eac-a790-a85d7239210a","Type":"ContainerDied","Data":"eef1699f596589ae3fbd470fd18f58fc164cc41690252619d2132ccad647342c"}
Sep 10 23:23:49.985275 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.985071    2565 scope.go:117] "RemoveContainer" containerID="eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"
Sep 10 23:23:49.995062 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.995041    2565 scope.go:117] "RemoveContainer" containerID="eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"
Sep 10 23:23:49.995331 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:23:49.995312    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25\": container with ID starting with eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25 not found: ID does not exist" containerID="eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"
Sep 10 23:23:49.995408 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:49.995338    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25"} err="failed to get container status \"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25\": rpc error: code = NotFound desc = could not find container \"eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25\": container with ID starting with eabe103c3cdda1103bc312b77edb2dd1b03d801a16c2b52026bb809f68604f25 not found: ID does not exist"
Sep 10 23:23:50.006036 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:50.006013    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:23:50.010589 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:50.010558    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/sub-test-model-4b268f-kserve-dc5f4745f-8czcr"]
Sep 10 23:23:50.210472 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:50.210437    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5973219e-e60b-4eac-a790-a85d7239210a" path="/var/lib/kubelet/pods/5973219e-e60b-4eac-a790-a85d7239210a/volumes"
Sep 10 23:23:50.210843 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:50.210830    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d789d6c8-c21a-468a-9695-c9a28a4d6172" path="/var/lib/kubelet/pods/d789d6c8-c21a-468a-9695-c9a28a4d6172/volumes"
Sep 10 23:23:55.655149 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:23:55.655111    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:24:07.161192 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161159    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:24:07.161614 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161582    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" containerName="maas-api"
Sep 10 23:24:07.161614 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161596    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" containerName="maas-api"
Sep 10 23:24:07.161614 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161609    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" containerName="test-curl-7658-a751ee"
Sep 10 23:24:07.161614 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161615    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" containerName="test-curl-7658-a751ee"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161624    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5973219e-e60b-4eac-a790-a85d7239210a" containerName="main"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161633    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5973219e-e60b-4eac-a790-a85d7239210a" containerName="main"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161659    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d789d6c8-c21a-468a-9695-c9a28a4d6172" containerName="main"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161665    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d789d6c8-c21a-468a-9695-c9a28a4d6172" containerName="main"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161735    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a6dcc0-0635-4dfa-bd4d-a3feb5257e00" containerName="maas-api"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161743    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="14651d06-0af9-4a82-ad4b-ae9e0dc825e2" containerName="test-curl-7658-a751ee"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161752    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d789d6c8-c21a-468a-9695-c9a28a4d6172" containerName="main"
Sep 10 23:24:07.161832 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.161758    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5973219e-e60b-4eac-a790-a85d7239210a" containerName="main"
Sep 10 23:24:07.166618 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.166599    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.168750 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.168732    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"kube-root-ca.crt\""
Sep 10 23:24:07.169512 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.169493    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"rate-test-model-4b268f-kserve-self-signed-certs\""
Sep 10 23:24:07.169633 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.169493    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"default-dockercfg-pfgst\""
Sep 10 23:24:07.169633 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.169531    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-4b268f\"/\"openshift-service-ca.crt\""
Sep 10 23:24:07.173666 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.173645    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:24:07.268184 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268154    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.268184 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268192    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.268428 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcmrv\" (UniqueName: \"kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.268468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268434    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.268515 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268464    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.268515 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.268491    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.369889 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.369852    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcmrv\" (UniqueName: \"kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.369908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.369937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.369961    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.370019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370080 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.370058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370389 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.370328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370389 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.370377    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.370484 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.370447    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.372444 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.372414    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.372746 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.372731    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.378773 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.378741    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcmrv\" (UniqueName: \"kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv\") pod \"rate-test-model-4b268f-kserve-866f4d95f9-9zs4t\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") " pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.477135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.477093    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:07.748965 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:07.748933    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:24:07.750698 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:24:07.750667    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b70af38_c407_458f_bf4d_fd7864c98263.slice/crio-e08848cf432b1e33e970cf876eb98e7178fcaae575750206ceea45facdb6b42b WatchSource:0}: Error finding container e08848cf432b1e33e970cf876eb98e7178fcaae575750206ceea45facdb6b42b: Status 404 returned error can't find the container with id e08848cf432b1e33e970cf876eb98e7178fcaae575750206ceea45facdb6b42b
Sep 10 23:24:08.077232 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.077127    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" event={"ID":"8b70af38-c407-458f-bf4d-fd7864c98263","Type":"ContainerStarted","Data":"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"}
Sep 10 23:24:08.077232 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.077182    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" event={"ID":"8b70af38-c407-458f-bf4d-fd7864c98263","Type":"ContainerStarted","Data":"e08848cf432b1e33e970cf876eb98e7178fcaae575750206ceea45facdb6b42b"}
Sep 10 23:24:08.095224 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.095179    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" podStartSLOduration=1.095163225 podStartE2EDuration="1.095163225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:24:08.093721756 +0000 UTC m=+1570.380138704" watchObservedRunningTime="2026-09-10 23:24:08.095163225 +0000 UTC m=+1570.381580172"
Sep 10 23:24:08.136539 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.136504    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:24:08.136827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.136781    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" podUID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" containerName="maas-api" containerID="cri-o://a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d" gracePeriod=30
Sep 10 23:24:08.377154 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.377124    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:24:08.479851 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.479813    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs\") pod \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") "
Sep 10 23:24:08.480039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.479861    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca\") pod \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") "
Sep 10 23:24:08.480039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.480012    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmvbn\" (UniqueName: \"kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn\") pod \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") "
Sep 10 23:24:08.480163 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.480045    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls\") pod \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\" (UID: \"9e9edc9d-71b5-4ed3-a7d4-9666034fa048\") "
Sep 10 23:24:08.480281 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.480248    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e9edc9d-71b5-4ed3-a7d4-9666034fa048" (UID: "9e9edc9d-71b5-4ed3-a7d4-9666034fa048"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:24:08.480433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.480407    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-openshift-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:08.482242 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.482213    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9e9edc9d-71b5-4ed3-a7d4-9666034fa048" (UID: "9e9edc9d-71b5-4ed3-a7d4-9666034fa048"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:24:08.482370 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.482284    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e9edc9d-71b5-4ed3-a7d4-9666034fa048" (UID: "9e9edc9d-71b5-4ed3-a7d4-9666034fa048"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:24:08.482447 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.482431    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn" (OuterVolumeSpecName: "kube-api-access-nmvbn") pod "9e9edc9d-71b5-4ed3-a7d4-9666034fa048" (UID: "9e9edc9d-71b5-4ed3-a7d4-9666034fa048"). InnerVolumeSpecName "kube-api-access-nmvbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:24:08.581024 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.580990    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmvbn\" (UniqueName: \"kubernetes.io/projected/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-kube-api-access-nmvbn\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:08.581024 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.581022    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-maas-api-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:08.581024 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:08.581032    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e9edc9d-71b5-4ed3-a7d4-9666034fa048-metrics-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:09.083649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.083615    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" containerID="a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d" exitCode=0
Sep 10 23:24:09.083836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.083658    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"
Sep 10 23:24:09.083836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.083698    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" event={"ID":"9e9edc9d-71b5-4ed3-a7d4-9666034fa048","Type":"ContainerDied","Data":"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"}
Sep 10 23:24:09.083836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.083750    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j" event={"ID":"9e9edc9d-71b5-4ed3-a7d4-9666034fa048","Type":"ContainerDied","Data":"450baad5a3e1962563124d8b0d1e252ebbe5e600e9fc17861378c48d48923bcb"}
Sep 10 23:24:09.083836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.083778    2565 scope.go:117] "RemoveContainer" containerID="a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"
Sep 10 23:24:09.093182 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.093165    2565 scope.go:117] "RemoveContainer" containerID="a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"
Sep 10 23:24:09.093470 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:24:09.093454    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d\": container with ID starting with a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d not found: ID does not exist" containerID="a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"
Sep 10 23:24:09.093533 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.093478    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d"} err="failed to get container status \"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d\": rpc error: code = NotFound desc = could not find container \"a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d\": container with ID starting with a214ceca85f9b0710f5eeab84228f888405e30c560c7276de6207a325cf4095d not found: ID does not exist"
Sep 10 23:24:09.107659 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.107632    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:24:09.113599 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:09.113572    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-44c67e-67956f76dd-g9g4j"]
Sep 10 23:24:10.210636 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:10.210596    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" path="/var/lib/kubelet/pods/9e9edc9d-71b5-4ed3-a7d4-9666034fa048/volumes"
Sep 10 23:24:14.561787 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:14.561752    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:24:14.830502 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:14.830414    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:24:14.830755 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:14.830727    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy" containerID="cri-o://0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d" gracePeriod=30
Sep 10 23:24:15.170839 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.170755    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"]
Sep 10 23:24:15.171169 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.171157    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" containerName="maas-api"
Sep 10 23:24:15.171212 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.171171    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" containerName="maas-api"
Sep 10 23:24:15.171265 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.171255    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9edc9d-71b5-4ed3-a7d4-9666034fa048" containerName="maas-api"
Sep 10 23:24:15.175751 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.175732    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:15.177876 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.177850    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:24:15.178273 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.178246    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:24:15.184119 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.183791    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"]
Sep 10 23:24:15.245381 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.245346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt\" (UID: \"a642f2ee-ab3f-4d08-b74f-281b630fd93a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:15.346417 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.346380    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt\" (UID: \"a642f2ee-ab3f-4d08-b74f-281b630fd93a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:15.346999 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.346979    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt\" (UID: \"a642f2ee-ab3f-4d08-b74f-281b630fd93a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:15.488692 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.488654    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:15.763396 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:15.763368    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"]
Sep 10 23:24:15.765561 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:24:15.765534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda642f2ee_ab3f_4d08_b74f_281b630fd93a.slice/crio-3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2 WatchSource:0}: Error finding container 3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2: Status 404 returned error can't find the container with id 3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2
Sep 10 23:24:16.122454 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:16.122363    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt" event={"ID":"a642f2ee-ab3f-4d08-b74f-281b630fd93a","Type":"ContainerStarted","Data":"b08dc839f05b163064092afdb49e2a38822169c47f61cd291778815dd8a20f0a"}
Sep 10 23:24:16.122454 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:16.122418    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt" event={"ID":"a642f2ee-ab3f-4d08-b74f-281b630fd93a","Type":"ContainerStarted","Data":"3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2"}
Sep 10 23:24:16.138088 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:16.138042    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt" podStartSLOduration=1.138027296 podStartE2EDuration="1.138027296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:24:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:24:16.136484849 +0000 UTC m=+1578.422901797" watchObservedRunningTime="2026-09-10 23:24:16.138027296 +0000 UTC m=+1578.424444243"
Sep 10 23:24:17.129409 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:17.129373    2565 generic.go:356] "Generic (PLEG): container finished" podID="a642f2ee-ab3f-4d08-b74f-281b630fd93a" containerID="b08dc839f05b163064092afdb49e2a38822169c47f61cd291778815dd8a20f0a" exitCode=0
Sep 10 23:24:17.129786 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:17.129462    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt" event={"ID":"a642f2ee-ab3f-4d08-b74f-281b630fd93a","Type":"ContainerDied","Data":"b08dc839f05b163064092afdb49e2a38822169c47f61cd291778815dd8a20f0a"}
Sep 10 23:24:17.477615 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:17.477573    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:17.477868 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:17.477842    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:17.491010 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:17.490983    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:18.146196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:18.146168    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:24:18.272008 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:18.271979    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:18.379826 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:18.379796    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca\") pod \"a642f2ee-ab3f-4d08-b74f-281b630fd93a\" (UID: \"a642f2ee-ab3f-4d08-b74f-281b630fd93a\") "
Sep 10 23:24:18.380174 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:18.380150    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a642f2ee-ab3f-4d08-b74f-281b630fd93a" (UID: "a642f2ee-ab3f-4d08-b74f-281b630fd93a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:24:18.480494 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:18.480457    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a642f2ee-ab3f-4d08-b74f-281b630fd93a-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:19.140662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:19.140623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt" event={"ID":"a642f2ee-ab3f-4d08-b74f-281b630fd93a","Type":"ContainerDied","Data":"3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2"}
Sep 10 23:24:19.140662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:19.140646    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"
Sep 10 23:24:19.140889 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:19.140681    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a6ad247d0577127ed29bce4d645ae1802fdf9f74cf7192480a5ba62f484d8f2"
Sep 10 23:24:20.092990 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.092964    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:24:20.147520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.147434    2565 generic.go:356] "Generic (PLEG): container finished" podID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerID="0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d" exitCode=0
Sep 10 23:24:20.147520 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.147485    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"
Sep 10 23:24:20.147701 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.147512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerDied","Data":"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"}
Sep 10 23:24:20.147701 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.147568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z" event={"ID":"63352a3b-05f7-47ec-9d4f-9609519bbfc7","Type":"ContainerDied","Data":"67531507d43e95a04db5d5bc25e054933c72c6235adeec81e34fc79803aa2c7f"}
Sep 10 23:24:20.147701 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.147590    2565 scope.go:117] "RemoveContainer" containerID="0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"
Sep 10 23:24:20.156609 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.156591    2565 scope.go:117] "RemoveContainer" containerID="34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"
Sep 10 23:24:20.164669 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.164648    2565 scope.go:117] "RemoveContainer" containerID="0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"
Sep 10 23:24:20.164966 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:24:20.164945    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d\": container with ID starting with 0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d not found: ID does not exist" containerID="0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"
Sep 10 23:24:20.165056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.164974    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d"} err="failed to get container status \"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d\": rpc error: code = NotFound desc = could not find container \"0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d\": container with ID starting with 0f50c62cef2d6d6c06d4979786812832716101012ab95cb7e95e27241c69cf6d not found: ID does not exist"
Sep 10 23:24:20.165056 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.164993    2565 scope.go:117] "RemoveContainer" containerID="34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"
Sep 10 23:24:20.165282 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:24:20.165264    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6\": container with ID starting with 34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6 not found: ID does not exist" containerID="34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"
Sep 10 23:24:20.165369 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.165313    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6"} err="failed to get container status \"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6\": rpc error: code = NotFound desc = could not find container \"34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6\": container with ID starting with 34f8c1ebe4c83b4b79f50f62d5f68c6cd90e0f3a246eceaf4c28f73484330aa6 not found: ID does not exist"
Sep 10 23:24:20.195952 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.195923    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196078 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.195961    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196078 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.195984    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196078 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196004    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd4px\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196078 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196052    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196274 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196103    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196274 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196274 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196231    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:24:20.196468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196266    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:24:20.196468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196333    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196391    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs\") pod \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\" (UID: \"63352a3b-05f7-47ec-9d4f-9609519bbfc7\") "
Sep 10 23:24:20.196468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196424    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data" (OuterVolumeSpecName: "istio-data") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:24:20.196678 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196573    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:24:20.196749 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.196726    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:24:20.197043 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.197019    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-data\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.197135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.197048    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istiod-ca-cert\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.197135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.197062    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.197135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.197073    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-credential-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.197135 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.197087    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-workload-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.198583 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.198556    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token" (OuterVolumeSpecName: "istio-token") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:24:20.198693 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.198653    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:24:20.198870 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.198853    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px" (OuterVolumeSpecName: "kube-api-access-dd4px") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "kube-api-access-dd4px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:24:20.198942 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.198929    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63352a3b-05f7-47ec-9d4f-9609519bbfc7" (UID: "63352a3b-05f7-47ec-9d4f-9609519bbfc7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:24:20.297975 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.297935    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-podinfo\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.297975 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.297968    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-envoy\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.297975 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.297977    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd4px\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-kube-api-access-dd4px\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.298212 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.297987    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63352a3b-05f7-47ec-9d4f-9609519bbfc7-istio-token\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:20.464853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.464817    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:24:20.466608 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:20.466588    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-44c67e-openshift-default-58c799c87b-2rj5z"]
Sep 10 23:24:22.214492 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:22.212180    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" path="/var/lib/kubelet/pods/63352a3b-05f7-47ec-9d4f-9609519bbfc7/volumes"
Sep 10 23:24:25.801363 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:25.801329    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"]
Sep 10 23:24:25.804301 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:25.804265    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-94f305-6vzbt"]
Sep 10 23:24:26.209989 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:26.209955    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a642f2ee-ab3f-4d08-b74f-281b630fd93a" path="/var/lib/kubelet/pods/a642f2ee-ab3f-4d08-b74f-281b630fd93a/volumes"
Sep 10 23:24:31.950448 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:31.950413    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:24:32.892124 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892093    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"]
Sep 10 23:24:32.892511 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892491    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:24:32.892511 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892507    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892525    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892531    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892541    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a642f2ee-ab3f-4d08-b74f-281b630fd93a" containerName="revoke-keys"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892546    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a642f2ee-ab3f-4d08-b74f-281b630fd93a" containerName="revoke-keys"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892612    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a642f2ee-ab3f-4d08-b74f-281b630fd93a" containerName="revoke-keys"
Sep 10 23:24:32.892667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.892620    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:24:32.895480 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.895464    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:32.900636 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.900614    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:24:32.900770 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.900615    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:24:32.910529 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:32.910505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"]
Sep 10 23:24:33.028377 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:33.028338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n\" (UID: \"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:33.129469 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:33.129435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n\" (UID: \"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:33.130049 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:33.130029    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n\" (UID: \"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:33.205449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:33.205418    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:33.477418 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:33.477394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"]
Sep 10 23:24:33.479425 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:24:33.479396    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8259d5a6_f7ce_4f5f_a7b1_e137ba6b2e14.slice/crio-c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45 WatchSource:0}: Error finding container c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45: Status 404 returned error can't find the container with id c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45
Sep 10 23:24:34.221923 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:34.221890    2565 generic.go:356] "Generic (PLEG): container finished" podID="8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" containerID="1f79e3722c2f4292ad8f61e8379c1134fbfc02376d60f5e7870add3338df7688" exitCode=0
Sep 10 23:24:34.222271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:34.221970    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n" event={"ID":"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14","Type":"ContainerDied","Data":"1f79e3722c2f4292ad8f61e8379c1134fbfc02376d60f5e7870add3338df7688"}
Sep 10 23:24:34.222271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:34.222012    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n" event={"ID":"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14","Type":"ContainerStarted","Data":"c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45"}
Sep 10 23:24:35.358480 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:35.358452    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:35.453935 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:35.453892    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca\") pod \"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14\" (UID: \"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14\") "
Sep 10 23:24:35.454258 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:35.454235    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" (UID: "8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:24:35.554981 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:35.554899    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:24:36.232478 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:36.232398    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"
Sep 10 23:24:36.232478 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:36.232442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n" event={"ID":"8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14","Type":"ContainerDied","Data":"c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45"}
Sep 10 23:24:36.232660 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:36.232497    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c701dfa64e939ecb487e62d965d353678024bbd50db80e639a488bc5e78b3e45"
Sep 10 23:24:43.057850 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:43.057813    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:24:43.610670 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:43.610631    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"]
Sep 10 23:24:43.613481 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:43.613454    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-063e1d-j6b7n"]
Sep 10 23:24:44.210633 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:44.210599    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" path="/var/lib/kubelet/pods/8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14/volumes"
Sep 10 23:24:53.856650 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:24:53.856568    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:25:01.147331 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:01.147270    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:25:04.461324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:04.459594    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:25:06.288067 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.288030    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:25:06.288614 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.288595    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" containerName="revoke-keys"
Sep 10 23:25:06.288700 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.288616    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" containerName="revoke-keys"
Sep 10 23:25:06.288759 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.288750    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="63352a3b-05f7-47ec-9d4f-9609519bbfc7" containerName="istio-proxy"
Sep 10 23:25:06.288809 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.288770    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8259d5a6-f7ce-4f5f-a7b1-e137ba6b2e14" containerName="revoke-keys"
Sep 10 23:25:06.293345 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.293324    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.295780 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.295747    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:25:06.295780 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.295764    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 23:25:06.295780 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.295749    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 23:25:06.296392 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.296369    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-595479-serving-cert\""
Sep 10 23:25:06.296500 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.296376    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9dl9w\""
Sep 10 23:25:06.302139 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.302114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:25:06.340928 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.340895    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.340928 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.340930    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsvk5\" (UniqueName: \"kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.341134 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.340951    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.341134 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.340973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.442332 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.442246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.442533 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.442348    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.442533 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.442510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.442659 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.442543    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsvk5\" (UniqueName: \"kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.443178 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.443156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.444934 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.444910    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.445036 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.444933    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.450574 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.450555    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsvk5\" (UniqueName: \"kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5\") pod \"maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.605189 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.605115    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:06.879335 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.879303    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:25:06.881514 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:25:06.881487    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20077ea_e902_42f4_8758_8104c6af07a5.slice/crio-30c49adb281a711a1fb5189062ba840ebb46976a1374f8f4781af5d458e8d82d WatchSource:0}: Error finding container 30c49adb281a711a1fb5189062ba840ebb46976a1374f8f4781af5d458e8d82d: Status 404 returned error can't find the container with id 30c49adb281a711a1fb5189062ba840ebb46976a1374f8f4781af5d458e8d82d
Sep 10 23:25:06.882903 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:06.882882    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:25:07.396802 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:07.396766    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" event={"ID":"e20077ea-e902-42f4-8758-8104c6af07a5","Type":"ContainerStarted","Data":"30c49adb281a711a1fb5189062ba840ebb46976a1374f8f4781af5d458e8d82d"}
Sep 10 23:25:08.403838 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:08.403798    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" event={"ID":"e20077ea-e902-42f4-8758-8104c6af07a5","Type":"ContainerStarted","Data":"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"}
Sep 10 23:25:08.404234 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:08.403935    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:08.420636 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:08.420592    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" podStartSLOduration=1.775804747 podStartE2EDuration="2.42057845s" totalImagesPullingTime="644.773703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:25:08.419975705 +0000 UTC m=+1630.706392652" watchObservedRunningTime="2026-09-10 23:25:08.42057845 +0000 UTC m=+1630.706995395"
Sep 10 23:25:14.413848 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:14.413818    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:25:15.778624 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:15.778591    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:25:16.167234 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.167123    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" podUID="8b70af38-c407-458f-bf4d-fd7864c98263" containerName="main" containerID="cri-o://4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f" gracePeriod=45
Sep 10 23:25:16.404999 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.404974    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:25:16.443952 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.443919    2565 generic.go:356] "Generic (PLEG): container finished" podID="8b70af38-c407-458f-bf4d-fd7864c98263" containerID="4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f" exitCode=0
Sep 10 23:25:16.444140 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.443965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"
Sep 10 23:25:16.444140 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.444006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" event={"ID":"8b70af38-c407-458f-bf4d-fd7864c98263","Type":"ContainerDied","Data":"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"}
Sep 10 23:25:16.444140 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.444053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t" event={"ID":"8b70af38-c407-458f-bf4d-fd7864c98263","Type":"ContainerDied","Data":"e08848cf432b1e33e970cf876eb98e7178fcaae575750206ceea45facdb6b42b"}
Sep 10 23:25:16.444140 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.444076    2565 scope.go:117] "RemoveContainer" containerID="4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"
Sep 10 23:25:16.452704 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.452687    2565 scope.go:117] "RemoveContainer" containerID="4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"
Sep 10 23:25:16.452991 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:25:16.452974    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f\": container with ID starting with 4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f not found: ID does not exist" containerID="4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"
Sep 10 23:25:16.453057 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.453016    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f"} err="failed to get container status \"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f\": rpc error: code = NotFound desc = could not find container \"4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f\": container with ID starting with 4b3501a90c9040d00c7e08e653b3f0267562259fa073e01d62b68c254d3e071f not found: ID does not exist"
Sep 10 23:25:16.543091 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543062    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543284 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543103    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcmrv\" (UniqueName: \"kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543284 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543125    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543284 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543143    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543504 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543279    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543504 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543414    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache\") pod \"8b70af38-c407-458f-bf4d-fd7864c98263\" (UID: \"8b70af38-c407-458f-bf4d-fd7864c98263\") "
Sep 10 23:25:16.543504 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543449    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:25:16.543662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543532    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home" (OuterVolumeSpecName: "home") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:25:16.543709 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543692    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:25:16.543850 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.543825    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-tmp-dir\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.544094 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.544079    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-home\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.544138 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.544101    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-model-cache\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.545510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.545484    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm" (OuterVolumeSpecName: "dshm") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:25:16.545510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.545499    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv" (OuterVolumeSpecName: "kube-api-access-pcmrv") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "kube-api-access-pcmrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:25:16.545685 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.545559    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b70af38-c407-458f-bf4d-fd7864c98263" (UID: "8b70af38-c407-458f-bf4d-fd7864c98263"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:25:16.645028 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.644998    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b70af38-c407-458f-bf4d-fd7864c98263-tls-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.645028 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.645025    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcmrv\" (UniqueName: \"kubernetes.io/projected/8b70af38-c407-458f-bf4d-fd7864c98263-kube-api-access-pcmrv\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.645028 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.645034    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b70af38-c407-458f-bf4d-fd7864c98263-dshm\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:25:16.765448 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.765414    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:25:16.769241 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:16.769215    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-4b268f/rate-test-model-4b268f-kserve-866f4d95f9-9zs4t"]
Sep 10 23:25:18.209674 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:18.209642    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b70af38-c407-458f-bf4d-fd7864c98263" path="/var/lib/kubelet/pods/8b70af38-c407-458f-bf4d-fd7864c98263/volumes"
Sep 10 23:25:30.945627 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.945594    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:25:30.946084 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.945980    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b70af38-c407-458f-bf4d-fd7864c98263" containerName="main"
Sep 10 23:25:30.946084 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.945990    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b70af38-c407-458f-bf4d-fd7864c98263" containerName="main"
Sep 10 23:25:30.946084 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.946065    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b70af38-c407-458f-bf4d-fd7864c98263" containerName="main"
Sep 10 23:25:30.949159 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.949143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.951488 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.951465    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 23:25:30.951621 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.951489    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-c6c079-openshift-default-dockercfg-lz8vj\""
Sep 10 23:25:30.962532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.962509    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:25:30.974177 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974151    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974370 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974198    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974370 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974272    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974370 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9xq\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974399    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974420    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974490 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974481    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:30.974622 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:30.974517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082073 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082449 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082480    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9xq\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082531    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082568    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082649 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082604    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.082827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082780    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.083000 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.082971    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.083089 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.083037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.083237 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.083206    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.083929 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.083907    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.085199 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.085180    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.085336 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.085314    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.091033 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.091008    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.091117 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.091091    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9xq\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq\") pod \"e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") " pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.259228 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.259139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:31.535218 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.535190    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:25:31.537449 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:25:31.537415    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e7b0d5c_a794_41b9_a6b2_02817f0a2f5a.slice/crio-4bf0070b503fef7fcec4838c2c311b43774dc5ac5167e71ccaad60251d8b529c WatchSource:0}: Error finding container 4bf0070b503fef7fcec4838c2c311b43774dc5ac5167e71ccaad60251d8b529c: Status 404 returned error can't find the container with id 4bf0070b503fef7fcec4838c2c311b43774dc5ac5167e71ccaad60251d8b529c
Sep 10 23:25:31.540017 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.539986    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:31.540071 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.540049    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:31.540104 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:31.540076    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:32.529877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:32.529844    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerStarted","Data":"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"}
Sep 10 23:25:32.530245 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:32.529893    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerStarted","Data":"4bf0070b503fef7fcec4838c2c311b43774dc5ac5167e71ccaad60251d8b529c"}
Sep 10 23:25:32.551330 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:32.551260    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podStartSLOduration=2.551241954 podStartE2EDuration="2.551241954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:25:32.549892849 +0000 UTC m=+1654.836309796" watchObservedRunningTime="2026-09-10 23:25:32.551241954 +0000 UTC m=+1654.837658901"
Sep 10 23:25:33.259730 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:33.259693    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:33.264658 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:33.264632    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:33.535361 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:33.535241    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:33.536175 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:33.536159    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:41.664585 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:41.664541    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:25:49.577510 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:49.577461    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 23:25:49.577978 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:49.577574    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 23:25:51.637010 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.636912    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerID="53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03" exitCode=0
Sep 10 23:25:51.637010 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.636973    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerDied","Data":"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"}
Sep 10 23:25:51.637487 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.637431    2565 scope.go:117] "RemoveContainer" containerID="53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"
Sep 10 23:25:51.638515 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.638484    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:51.638630 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.638570    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:51.638630 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:51.638620    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:25:52.645683 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:52.645645    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerStarted","Data":"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"}
Sep 10 23:25:53.260071 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:53.260030    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:53.260254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:53.260232    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:53.260329 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:53.260277    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:54.259586 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:54.259545    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:54.260001 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:54.259601    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:55.260278 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:55.260235    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:55.260681 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:55.260318    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:56.259729 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:56.259685    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:56.259956 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:56.259747    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:57.259980 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:57.259945    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:57.260396 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:57.259998    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:58.259802 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:58.259764    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:25:58.259982 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:58.259832    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:25:59.263218 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:59.263181    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:59.263620 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:59.263584    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:25:59.264192 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:25:59.264178    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:26:20.056754 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:26:20.056667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:26:54.455559 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:26:54.455529    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:27:01.455765 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.455734    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:01.460375 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.460343    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.463522 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.463497    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-2cdd19-openshift-default-dockercfg-ms8mr\""
Sep 10 23:27:01.474434 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.474413    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:01.556422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556425    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556469    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2rft\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556691 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556890 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.556890 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.556780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658258    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658324 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658592 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658396    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658592 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658592 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658603    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2rft\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658782 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658733    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658998 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658786    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658998 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658817    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.658998 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.658971    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.659125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.659104    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.659246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.659224    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.660851 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.660831    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.661115 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.661095    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.666754 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.666732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.666754 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.666743    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2rft\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft\") pod \"e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") " pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:01.774462 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:01.774369    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:02.058711 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:02.058688    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:02.061064 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:27:02.060994    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf197b3_c7a5_4ca8_bb75_c45f077948f9.slice/crio-65da8d0c39a023ac1dbdd121583250ef4f0469fa2690c0e5d757f7c9053f3548 WatchSource:0}: Error finding container 65da8d0c39a023ac1dbdd121583250ef4f0469fa2690c0e5d757f7c9053f3548: Status 404 returned error can't find the container with id 65da8d0c39a023ac1dbdd121583250ef4f0469fa2690c0e5d757f7c9053f3548
Sep 10 23:27:02.063271 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:02.063234    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:02.063410 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:02.063355    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:02.063410 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:02.063392    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:03.010436 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:03.010397    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerStarted","Data":"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"}
Sep 10 23:27:03.010859 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:03.010451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerStarted","Data":"65da8d0c39a023ac1dbdd121583250ef4f0469fa2690c0e5d757f7c9053f3548"}
Sep 10 23:27:03.038836 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:03.038782    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podStartSLOduration=2.038767663 podStartE2EDuration="2.038767663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:27:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:27:03.036404327 +0000 UTC m=+1745.322821273" watchObservedRunningTime="2026-09-10 23:27:03.038767663 +0000 UTC m=+1745.325184610"
Sep 10 23:27:03.775229 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:03.775192    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:03.780129 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:03.780103    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:04.016167 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:04.016133    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:04.017596 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:04.017572    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:13.868131 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:13.868095    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:27:25.138764 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.138735    2565 generic.go:356] "Generic (PLEG): container finished" podID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerID="bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa" exitCode=0
Sep 10 23:27:25.139270 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.138813    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerDied","Data":"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"}
Sep 10 23:27:25.139270 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.139202    2565 scope.go:117] "RemoveContainer" containerID="bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"
Sep 10 23:27:25.140413 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.140389    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:25.140475 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.140452    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:25.140515 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:25.140481    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 10 23:27:26.146276 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:26.146231    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerStarted","Data":"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"}
Sep 10 23:27:26.775453 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:26.775416    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:26.775656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:26.775607    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:27:26.775736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:26.775674    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:27:27.774808 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:27.774769    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:27:27.775198 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:27.774822    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:27:28.775604 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:28.775571    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:27:28.775986 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:28.775624    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:27:29.775609 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:29.775572    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:27:29.775997 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:29.775628    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:27:30.775053 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:30.775017    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:27:30.775245 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:30.775077    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:27:31.779494 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:31.779462    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:31.779969 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:31.779844    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:31.780544 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:31.780524    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:41.352433 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:41.352394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:27:43.895664 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:43.895631    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:43.896081 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:43.895854    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" containerID="cri-o://b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f" gracePeriod=30
Sep 10 23:27:46.782443 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:46.782405    2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:27:46.782827 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:46.782463    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:27:49.144696 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.144671    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:49.219067 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219032    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219082    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219111    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219144    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219172    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219339 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219243    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219345    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2rft\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219386    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219412    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert\") pod \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\" (UID: \"7bf197b3-c7a5-4ca8-bb75-c45f077948f9\") "
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219524    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219537    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:27:49.219687 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219627    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:27:49.219957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219825    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:27:49.219957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219837    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-credential-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.219957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219853    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.219957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219863    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-data\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.219957 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.219876    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-workload-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.221804 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.221783    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:27:49.221856 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.221813    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft" (OuterVolumeSpecName: "kube-api-access-q2rft") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "kube-api-access-q2rft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:27:49.222009 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.221995    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:27:49.222075 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.222059    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bf197b3-c7a5-4ca8-bb75-c45f077948f9" (UID: "7bf197b3-c7a5-4ca8-bb75-c45f077948f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:27:49.264869 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.264837    2565 generic.go:356] "Generic (PLEG): container finished" podID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerID="b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f" exitCode=0
Sep 10 23:27:49.265038 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.264871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerDied","Data":"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"}
Sep 10 23:27:49.265038 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.264891    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"
Sep 10 23:27:49.265038 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.264927    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp" event={"ID":"7bf197b3-c7a5-4ca8-bb75-c45f077948f9","Type":"ContainerDied","Data":"65da8d0c39a023ac1dbdd121583250ef4f0469fa2690c0e5d757f7c9053f3548"}
Sep 10 23:27:49.265038 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.264950    2565 scope.go:117] "RemoveContainer" containerID="b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"
Sep 10 23:27:49.274578 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.274561    2565 scope.go:117] "RemoveContainer" containerID="bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"
Sep 10 23:27:49.282547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.282524    2565 scope.go:117] "RemoveContainer" containerID="b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"
Sep 10 23:27:49.282812 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:27:49.282794    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f\": container with ID starting with b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f not found: ID does not exist" containerID="b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"
Sep 10 23:27:49.282875 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.282822    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f"} err="failed to get container status \"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f\": rpc error: code = NotFound desc = could not find container \"b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f\": container with ID starting with b1d3d8a5f43823c88c815592d42e42e5cd284486a9fd68f73d6bd2301b28829f not found: ID does not exist"
Sep 10 23:27:49.282875 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.282842    2565 scope.go:117] "RemoveContainer" containerID="bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"
Sep 10 23:27:49.283080 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:27:49.283062    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa\": container with ID starting with bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa not found: ID does not exist" containerID="bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"
Sep 10 23:27:49.283140 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.283090    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa"} err="failed to get container status \"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa\": rpc error: code = NotFound desc = could not find container \"bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa\": container with ID starting with bee6e735d692d6ce97f4d78a00186d132b3158dc59de56d5eb32fb7be29fcbfa not found: ID does not exist"
Sep 10 23:27:49.287736 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.287707    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:49.290075 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.290052    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2cdd19-openshift-default-6ff6c577d-ldqdp"]
Sep 10 23:27:49.321269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.321241    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-token\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.321269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.321265    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-envoy\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.321269 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.321273    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istio-podinfo\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.321446 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.321282    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2rft\" (UniqueName: \"kubernetes.io/projected/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-kube-api-access-q2rft\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:49.321446 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:49.321313    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bf197b3-c7a5-4ca8-bb75-c45f077948f9-istiod-ca-cert\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:27:50.210751 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:27:50.210718    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" path="/var/lib/kubelet/pods/7bf197b3-c7a5-4ca8-bb75-c45f077948f9/volumes"
Sep 10 23:28:00.556495 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:00.556448    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:28:02.943281 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:02.943243    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:28:02.943732 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:02.943526    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" podUID="e20077ea-e902-42f4-8758-8104c6af07a5" containerName="maas-api" containerID="cri-o://493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3" gracePeriod=30
Sep 10 23:28:03.198415 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.198358    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:28:03.334944 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.334909    2565 generic.go:356] "Generic (PLEG): container finished" podID="e20077ea-e902-42f4-8758-8104c6af07a5" containerID="493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3" exitCode=0
Sep 10 23:28:03.335125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.334973    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"
Sep 10 23:28:03.335125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.335000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" event={"ID":"e20077ea-e902-42f4-8758-8104c6af07a5","Type":"ContainerDied","Data":"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"}
Sep 10 23:28:03.335125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.335054    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x" event={"ID":"e20077ea-e902-42f4-8758-8104c6af07a5","Type":"ContainerDied","Data":"30c49adb281a711a1fb5189062ba840ebb46976a1374f8f4781af5d458e8d82d"}
Sep 10 23:28:03.335125 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.335077    2565 scope.go:117] "RemoveContainer" containerID="493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"
Sep 10 23:28:03.345821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.345803    2565 scope.go:117] "RemoveContainer" containerID="493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"
Sep 10 23:28:03.346155 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:28:03.346122    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3\": container with ID starting with 493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3 not found: ID does not exist" containerID="493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"
Sep 10 23:28:03.346260 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.346167    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3"} err="failed to get container status \"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3\": rpc error: code = NotFound desc = could not find container \"493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3\": container with ID starting with 493cc6f99fc8b69849cbf7d7e9eea976d1f94d99a75e4f22bd910acccc8558a3 not found: ID does not exist"
Sep 10 23:28:03.348864 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.348845    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls\") pod \"e20077ea-e902-42f4-8758-8104c6af07a5\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") "
Sep 10 23:28:03.348961 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.348913    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs\") pod \"e20077ea-e902-42f4-8758-8104c6af07a5\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") "
Sep 10 23:28:03.348961 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.348951    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsvk5\" (UniqueName: \"kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5\") pod \"e20077ea-e902-42f4-8758-8104c6af07a5\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") "
Sep 10 23:28:03.349065 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.348999    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca\") pod \"e20077ea-e902-42f4-8758-8104c6af07a5\" (UID: \"e20077ea-e902-42f4-8758-8104c6af07a5\") "
Sep 10 23:28:03.350083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.349499    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e20077ea-e902-42f4-8758-8104c6af07a5" (UID: "e20077ea-e902-42f4-8758-8104c6af07a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:28:03.351564 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.351525    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e20077ea-e902-42f4-8758-8104c6af07a5" (UID: "e20077ea-e902-42f4-8758-8104c6af07a5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:28:03.351657 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.351595    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5" (OuterVolumeSpecName: "kube-api-access-lsvk5") pod "e20077ea-e902-42f4-8758-8104c6af07a5" (UID: "e20077ea-e902-42f4-8758-8104c6af07a5"). InnerVolumeSpecName "kube-api-access-lsvk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:28:03.351781 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.351764    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e20077ea-e902-42f4-8758-8104c6af07a5" (UID: "e20077ea-e902-42f4-8758-8104c6af07a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:28:03.449896 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.449821    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-maas-api-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:28:03.449896 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.449848    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e20077ea-e902-42f4-8758-8104c6af07a5-metrics-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:28:03.449896 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.449857    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsvk5\" (UniqueName: \"kubernetes.io/projected/e20077ea-e902-42f4-8758-8104c6af07a5-kube-api-access-lsvk5\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:28:03.449896 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.449867    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e20077ea-e902-42f4-8758-8104c6af07a5-openshift-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:28:03.656613 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.656582    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:28:03.660724 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:03.660700    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-595479-7cb96f7fbd-4xq5x"]
Sep 10 23:28:04.210667 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:04.210632    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20077ea-e902-42f4-8758-8104c6af07a5" path="/var/lib/kubelet/pods/e20077ea-e902-42f4-8758-8104c6af07a5/volumes"
Sep 10 23:28:10.059322 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:28:10.058542    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:29:07.548975 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.548938    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"]
Sep 10 23:29:07.549438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549414    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20077ea-e902-42f4-8758-8104c6af07a5" containerName="maas-api"
Sep 10 23:29:07.549438 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549429    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20077ea-e902-42f4-8758-8104c6af07a5" containerName="maas-api"
Sep 10 23:29:07.549532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549457    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549462    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549469    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549474    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549532 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549533    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549707 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549543    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf197b3-c7a5-4ca8-bb75-c45f077948f9" containerName="istio-proxy"
Sep 10 23:29:07.549707 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.549550    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20077ea-e902-42f4-8758-8104c6af07a5" containerName="maas-api"
Sep 10 23:29:07.553873 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.553856    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:07.556102 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.556078    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-79mcz\""
Sep 10 23:29:07.556203 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.556078    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:29:07.559769 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.559742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"]
Sep 10 23:29:07.645778 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.645751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h\" (UID: \"614c8019-230b-4654-a961-4aa89c4a5916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:07.746930 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.746895    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h\" (UID: \"614c8019-230b-4654-a961-4aa89c4a5916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:07.747586 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.747568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h\" (UID: \"614c8019-230b-4654-a961-4aa89c4a5916\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:07.865710 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:07.865626    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:08.137378 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:08.137347    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"]
Sep 10 23:29:08.139733 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:29:08.139706    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614c8019_230b_4654_a961_4aa89c4a5916.slice/crio-4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7 WatchSource:0}: Error finding container 4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7: Status 404 returned error can't find the container with id 4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7
Sep 10 23:29:08.666083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:08.665998    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h" event={"ID":"614c8019-230b-4654-a961-4aa89c4a5916","Type":"ContainerStarted","Data":"ef2f82765fd7f037377540feee12594bf89c315fd28be9da90d873e6a7b0f7db"}
Sep 10 23:29:08.666083 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:08.666057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h" event={"ID":"614c8019-230b-4654-a961-4aa89c4a5916","Type":"ContainerStarted","Data":"4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7"}
Sep 10 23:29:08.683169 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:08.683119    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h" podStartSLOduration=1.683104068 podStartE2EDuration="1.683104068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:29:08.680743808 +0000 UTC m=+1870.967160757" watchObservedRunningTime="2026-09-10 23:29:08.683104068 +0000 UTC m=+1870.969521035"
Sep 10 23:29:09.675055 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:09.675018    2565 generic.go:356] "Generic (PLEG): container finished" podID="614c8019-230b-4654-a961-4aa89c4a5916" containerID="ef2f82765fd7f037377540feee12594bf89c315fd28be9da90d873e6a7b0f7db" exitCode=0
Sep 10 23:29:09.675468 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:09.675100    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h" event={"ID":"614c8019-230b-4654-a961-4aa89c4a5916","Type":"ContainerDied","Data":"ef2f82765fd7f037377540feee12594bf89c315fd28be9da90d873e6a7b0f7db"}
Sep 10 23:29:10.808821 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:10.808797    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:10.878469 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:10.878438    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca\") pod \"614c8019-230b-4654-a961-4aa89c4a5916\" (UID: \"614c8019-230b-4654-a961-4aa89c4a5916\") "
Sep 10 23:29:10.878737 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:10.878718    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "614c8019-230b-4654-a961-4aa89c4a5916" (UID: "614c8019-230b-4654-a961-4aa89c4a5916"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:29:10.979805 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:10.979732    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/614c8019-230b-4654-a961-4aa89c4a5916-maas-api-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:11.687434 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:11.687396    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"
Sep 10 23:29:11.687605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:11.687396    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h" event={"ID":"614c8019-230b-4654-a961-4aa89c4a5916","Type":"ContainerDied","Data":"4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7"}
Sep 10 23:29:11.687605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:11.687524    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b0c4fb414897841fd93c16fa5b9d0b4919f45331bda3997ba4c87916a1597a7"
Sep 10 23:29:18.238376 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:18.238326    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"]
Sep 10 23:29:18.240459 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:18.240437    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c6c079-rbd6h"]
Sep 10 23:29:20.210939 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:20.210858    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614c8019-230b-4654-a961-4aa89c4a5916" path="/var/lib/kubelet/pods/614c8019-230b-4654-a961-4aa89c4a5916/volumes"
Sep 10 23:29:21.257522 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:21.257494    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:29:25.279516 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:25.279483    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:29:25.279993 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:25.279702    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" containerID="cri-o://2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102" gracePeriod=30
Sep 10 23:29:29.266108 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:29.266070    2565 patch_prober.go:28] interesting pod/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:29:29.266547 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:29.266140    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:29:30.534673 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.534643    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:29:30.556757 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556664    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.556757 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556723    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.556757 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556756    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556781    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx9xq\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556811    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556869    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556909    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556942    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.556957    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557020    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data\") pod \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\" (UID: \"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a\") "
Sep 10 23:29:30.557397 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557200    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:29:30.557397 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557365    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.557501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557384    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istiod-ca-cert\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.557501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557426    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:29:30.557658 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557620    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:29:30.557987 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.557782    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:29:30.559469 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.559442    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:29:30.559581 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.559555    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:29:30.560422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.560390    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:29:30.560422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.560409    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq" (OuterVolumeSpecName: "kube-api-access-xx9xq") pod "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" (UID: "2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a"). InnerVolumeSpecName "kube-api-access-xx9xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:29:30.658717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658681    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-workload-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658711    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-data\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658717 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658719    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-podinfo\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658728    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx9xq\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-kube-api-access-xx9xq\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658738    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-envoy\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658746    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-credential-socket\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.658964 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.658755    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a-istio-token\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:29:30.785602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.785569    2565 generic.go:356] "Generic (PLEG): container finished" podID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerID="2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102" exitCode=0
Sep 10 23:29:30.785770 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.785620    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"
Sep 10 23:29:30.785770 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.785653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerDied","Data":"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"}
Sep 10 23:29:30.785770 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.785714    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk" event={"ID":"2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a","Type":"ContainerDied","Data":"4bf0070b503fef7fcec4838c2c311b43774dc5ac5167e71ccaad60251d8b529c"}
Sep 10 23:29:30.785770 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.785738    2565 scope.go:117] "RemoveContainer" containerID="2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"
Sep 10 23:29:30.795455 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.795434    2565 scope.go:117] "RemoveContainer" containerID="53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"
Sep 10 23:29:30.804424 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.804398    2565 scope.go:117] "RemoveContainer" containerID="2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"
Sep 10 23:29:30.805060 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:29:30.805031    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102\": container with ID starting with 2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102 not found: ID does not exist" containerID="2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"
Sep 10 23:29:30.805183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.805075    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102"} err="failed to get container status \"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102\": rpc error: code = NotFound desc = could not find container \"2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102\": container with ID starting with 2d3b72a131707e601f927a7e4ebdec9e195ae03815c7baf74b0264f81f5b5102 not found: ID does not exist"
Sep 10 23:29:30.805183 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.805101    2565 scope.go:117] "RemoveContainer" containerID="53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"
Sep 10 23:29:30.805465 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:29:30.805439    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03\": container with ID starting with 53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03 not found: ID does not exist" containerID="53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"
Sep 10 23:29:30.805551 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.805464    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03"} err="failed to get container status \"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03\": rpc error: code = NotFound desc = could not find container \"53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03\": container with ID starting with 53aa9b193f84abb368b075d46633bcbb78b25154e59ae91bc33bd7ad4cd98f03 not found: ID does not exist"
Sep 10 23:29:30.807122 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.807069    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:29:30.812173 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:30.812152    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-c6c079-openshift-default-855654c4d-qw6vk"]
Sep 10 23:29:32.210242 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:32.210214    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" path="/var/lib/kubelet/pods/2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a/volumes"
Sep 10 23:29:39.455321 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:39.455272    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:29:57.714419 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:29:57.714381    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:30:08.384090 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:30:08.384059    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:30:59.214551 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:30:59.214473    2565 scope.go:117] "RemoveContainer" containerID="1f79e3722c2f4292ad8f61e8379c1134fbfc02376d60f5e7870add3338df7688"
Sep 10 23:30:59.223680 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:30:59.223656    2565 scope.go:117] "RemoveContainer" containerID="b08dc839f05b163064092afdb49e2a38822169c47f61cd291778815dd8a20f0a"
Sep 10 23:31:29.356412 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:31:29.356382    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:31:54.161352 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:31:54.161322    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:32:02.453526 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:32:02.453490    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:32:10.460161 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:32:10.460115    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:32:27.754469 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:32:27.754438    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:33:58.451943 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:33:58.451865    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:34:06.652406 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:34:06.652371    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:34:15.055210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:34:15.055171    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:34:25.056366 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:34:25.056333    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:34:47.452941 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:34:47.452903    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:23.057991 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:23.057902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:30.555008 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:30.554965    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:37.259766 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.259725    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:37.499143 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499106    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:37.499566 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499550    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614c8019-230b-4654-a961-4aa89c4a5916" containerName="revoke-keys"
Sep 10 23:35:37.499635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499567    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c8019-230b-4654-a961-4aa89c4a5916" containerName="revoke-keys"
Sep 10 23:35:37.499635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499576    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.499635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499581    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.499635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499595    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.499635 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499600    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.499815 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499663    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="614c8019-230b-4654-a961-4aa89c4a5916" containerName="revoke-keys"
Sep 10 23:35:37.499815 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499673    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.499815 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.499680    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e7b0d5c-a794-41b9-a6b2-02817f0a2f5a" containerName="istio-proxy"
Sep 10 23:35:37.502727 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.502710    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.505090 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.505068    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:35:37.506164 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.506144    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 23:35:37.506164 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.506159    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9dl9w\""
Sep 10 23:35:37.506358 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.506157    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 23:35:37.506358 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.506157    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 23:35:37.512831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.512770    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:37.587549 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.587511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.587549 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.587554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.587767 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.587642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxh9v\" (UniqueName: \"kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.587767 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.587683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.688758 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.688697    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.689006 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.688817    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxh9v\" (UniqueName: \"kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.689006 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.688853    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.689006 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.688919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.689579 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.689543    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.691467 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.691444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.691577 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.691489    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.697411 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.697391    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxh9v\" (UniqueName: \"kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v\") pod \"maas-api-7f7f8945f9-ptcfj\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:37.815325 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:37.815204    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:38.087708 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:38.087659    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:38.090037 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:35:38.090009    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808d31c7_9099_47e8_918c_21587bddd77b.slice/crio-6decd4be369ac696479f7f093908b7644e3d9ffe33e55023be2cf12beffe6b49 WatchSource:0}: Error finding container 6decd4be369ac696479f7f093908b7644e3d9ffe33e55023be2cf12beffe6b49: Status 404 returned error can't find the container with id 6decd4be369ac696479f7f093908b7644e3d9ffe33e55023be2cf12beffe6b49
Sep 10 23:35:38.091574 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:38.091554    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:35:38.608157 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:38.608124    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" event={"ID":"808d31c7-9099-47e8-918c-21587bddd77b","Type":"ContainerStarted","Data":"6decd4be369ac696479f7f093908b7644e3d9ffe33e55023be2cf12beffe6b49"}
Sep 10 23:35:39.614998 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:39.614961    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" event={"ID":"808d31c7-9099-47e8-918c-21587bddd77b","Type":"ContainerStarted","Data":"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"}
Sep 10 23:35:39.615456 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:39.615071    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:39.633127 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:39.633065    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" podStartSLOduration=1.57073397 podStartE2EDuration="2.633049213s" totalImagesPullingTime="1.062315243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:35:39.631192291 +0000 UTC m=+2261.917609239" watchObservedRunningTime="2026-09-10 23:35:39.633049213 +0000 UTC m=+2261.919466200"
Sep 10 23:35:45.159758 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.159714    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:45.626405 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.626373    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:45.716518 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.716478    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-gh27x"]
Sep 10 23:35:45.720418 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.720397    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.730561 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.730534    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-gh27x"]
Sep 10 23:35:45.868626 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.868582    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-maas-api-tls\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.868853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.868642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38beaffa-a538-449d-b9af-fcdf7d0e4541-openshift-service-ca\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.868853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.868757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsnct\" (UniqueName: \"kubernetes.io/projected/38beaffa-a538-449d-b9af-fcdf7d0e4541-kube-api-access-gsnct\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.868853 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.868821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-metrics-certs\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.970118 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.970077    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsnct\" (UniqueName: \"kubernetes.io/projected/38beaffa-a538-449d-b9af-fcdf7d0e4541-kube-api-access-gsnct\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.970323 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.970133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-metrics-certs\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.970323 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.970167    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-maas-api-tls\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.970323 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.970202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38beaffa-a538-449d-b9af-fcdf7d0e4541-openshift-service-ca\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.970846 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.970827    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38beaffa-a538-449d-b9af-fcdf7d0e4541-openshift-service-ca\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.972978 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.972957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-metrics-certs\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.973065 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.973001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38beaffa-a538-449d-b9af-fcdf7d0e4541-maas-api-tls\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:45.979379 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:45.979349    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsnct\" (UniqueName: \"kubernetes.io/projected/38beaffa-a538-449d-b9af-fcdf7d0e4541-kube-api-access-gsnct\") pod \"maas-api-559477f47-gh27x\" (UID: \"38beaffa-a538-449d-b9af-fcdf7d0e4541\") " pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:46.032448 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:46.032392    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:46.307459 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:46.307375    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-gh27x"]
Sep 10 23:35:46.310743 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:35:46.310703    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38beaffa_a538_449d_b9af_fcdf7d0e4541.slice/crio-19500cb82f2f5b7d69b84dfa7eb81c3a73b95d0397d0b9b2d3f68232126b2645 WatchSource:0}: Error finding container 19500cb82f2f5b7d69b84dfa7eb81c3a73b95d0397d0b9b2d3f68232126b2645: Status 404 returned error can't find the container with id 19500cb82f2f5b7d69b84dfa7eb81c3a73b95d0397d0b9b2d3f68232126b2645
Sep 10 23:35:46.660571 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:46.660529    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x" event={"ID":"38beaffa-a538-449d-b9af-fcdf7d0e4541","Type":"ContainerStarted","Data":"19500cb82f2f5b7d69b84dfa7eb81c3a73b95d0397d0b9b2d3f68232126b2645"}
Sep 10 23:35:47.666791 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:47.666754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x" event={"ID":"38beaffa-a538-449d-b9af-fcdf7d0e4541","Type":"ContainerStarted","Data":"4ba38d447b26119c0cf23a2d3df97dc797262c57f8167a892a79ac0380106502"}
Sep 10 23:35:47.667202 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:47.666859    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:47.686233 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:47.686175    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x" podStartSLOduration=2.385709781 podStartE2EDuration="2.68615985s" totalImagesPullingTime="300.450069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:35:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:35:47.683680247 +0000 UTC m=+2269.970097193" watchObservedRunningTime="2026-09-10 23:35:47.68615985 +0000 UTC m=+2269.972576844"
Sep 10 23:35:52.051746 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:52.051707    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:35:53.677808 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:53.677780    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-559477f47-gh27x"
Sep 10 23:35:53.725039 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:53.725005    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:53.725281 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:53.725252    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" podUID="808d31c7-9099-47e8-918c-21587bddd77b" containerName="maas-api" containerID="cri-o://ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e" gracePeriod=30
Sep 10 23:35:54.977485 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:54.977458    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:55.060612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.060524    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs\") pod \"808d31c7-9099-47e8-918c-21587bddd77b\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") "
Sep 10 23:35:55.060612 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.060604    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxh9v\" (UniqueName: \"kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v\") pod \"808d31c7-9099-47e8-918c-21587bddd77b\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") "
Sep 10 23:35:55.060822 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.060636    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls\") pod \"808d31c7-9099-47e8-918c-21587bddd77b\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") "
Sep 10 23:35:55.060822 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.060692    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca\") pod \"808d31c7-9099-47e8-918c-21587bddd77b\" (UID: \"808d31c7-9099-47e8-918c-21587bddd77b\") "
Sep 10 23:35:55.061156 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.061129    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "808d31c7-9099-47e8-918c-21587bddd77b" (UID: "808d31c7-9099-47e8-918c-21587bddd77b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:35:55.062830 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.062799    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "808d31c7-9099-47e8-918c-21587bddd77b" (UID: "808d31c7-9099-47e8-918c-21587bddd77b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:35:55.062830 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.062801    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v" (OuterVolumeSpecName: "kube-api-access-gxh9v") pod "808d31c7-9099-47e8-918c-21587bddd77b" (UID: "808d31c7-9099-47e8-918c-21587bddd77b"). InnerVolumeSpecName "kube-api-access-gxh9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:35:55.062973 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.062843    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "808d31c7-9099-47e8-918c-21587bddd77b" (UID: "808d31c7-9099-47e8-918c-21587bddd77b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:35:55.162196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.162148    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/808d31c7-9099-47e8-918c-21587bddd77b-openshift-service-ca\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:35:55.162196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.162191    2565 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-metrics-certs\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:35:55.162196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.162202    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxh9v\" (UniqueName: \"kubernetes.io/projected/808d31c7-9099-47e8-918c-21587bddd77b-kube-api-access-gxh9v\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:35:55.162196 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.162211    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/808d31c7-9099-47e8-918c-21587bddd77b-maas-api-tls\") on node \"ip-10-0-137-120.ec2.internal\" DevicePath \"\""
Sep 10 23:35:55.709430 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.709396    2565 generic.go:356] "Generic (PLEG): container finished" podID="808d31c7-9099-47e8-918c-21587bddd77b" containerID="ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e" exitCode=0
Sep 10 23:35:55.709616 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.709447    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"
Sep 10 23:35:55.709616 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.709453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" event={"ID":"808d31c7-9099-47e8-918c-21587bddd77b","Type":"ContainerDied","Data":"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"}
Sep 10 23:35:55.709616 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.709506    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj" event={"ID":"808d31c7-9099-47e8-918c-21587bddd77b","Type":"ContainerDied","Data":"6decd4be369ac696479f7f093908b7644e3d9ffe33e55023be2cf12beffe6b49"}
Sep 10 23:35:55.709616 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.709537    2565 scope.go:117] "RemoveContainer" containerID="ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"
Sep 10 23:35:55.719226 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.719207    2565 scope.go:117] "RemoveContainer" containerID="ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"
Sep 10 23:35:55.719536 ip-10-0-137-120 kubenswrapper[2565]: E0910 23:35:55.719514    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e\": container with ID starting with ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e not found: ID does not exist" containerID="ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"
Sep 10 23:35:55.719582 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.719548    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e"} err="failed to get container status \"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e\": rpc error: code = NotFound desc = could not find container \"ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e\": container with ID starting with ea21818e8c2c7e22ee1cbd1651fdd87f6d191edf0c40366eab95f9632eacb11e not found: ID does not exist"
Sep 10 23:35:55.730191 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.730169    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:55.734197 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:55.734176    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-ptcfj"]
Sep 10 23:35:56.210185 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:56.210148    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="808d31c7-9099-47e8-918c-21587bddd77b" path="/var/lib/kubelet/pods/808d31c7-9099-47e8-918c-21587bddd77b/volumes"
Sep 10 23:35:59.275030 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:35:59.275000    2565 scope.go:117] "RemoveContainer" containerID="ef2f82765fd7f037377540feee12594bf89c315fd28be9da90d873e6a7b0f7db"
Sep 10 23:36:00.649602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:00.649570    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:36:08.150013 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:08.149978    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-595v2"]
Sep 10 23:36:11.461752 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:11.461716    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c979f8d77-crf2j_5fc1ca76-9474-4173-a533-cfb06fb946c8/authorino/0.log"
Sep 10 23:36:17.452525 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:17.452493    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfbc5c4d-pc92l_f57bab69-4315-492d-b1e4-a99246d5999c/copy-manifests/0.log"
Sep 10 23:36:17.465761 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:17.465732    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfbc5c4d-pc92l_f57bab69-4315-492d-b1e4-a99246d5999c/manager/0.log"
Sep 10 23:36:17.913768 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:17.913699    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-559477f47-gh27x_38beaffa-a538-449d-b9af-fcdf7d0e4541/maas-api/0.log"
Sep 10 23:36:19.107170 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:19.107141    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c979f8d77-crf2j_5fc1ca76-9474-4173-a533-cfb06fb946c8/authorino/0.log"
Sep 10 23:36:19.218017 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:19.217986    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-9qvlr_394635a7-a40c-411e-83e3-81ac20c50416/manager/0.log"
Sep 10 23:36:19.680254 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:19.680222    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-595v2_8a0be99c-0a8d-4dd1-a720-4f57498d3c75/limitador/0.log"
Sep 10 23:36:20.491409 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:20.491381    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9b555765-hzxqd_966b510e-6894-4f2a-ba50-d51034a47d7f/kube-auth-proxy/0.log"
Sep 10 23:36:21.329242 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:21.329213    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-v86xc_ff0cc5d6-d73c-4518-a1f0-f73873d691de/main/0.log"
Sep 10 23:36:21.689894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:21.689867    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-z4zwt_b27ac81e-1254-4343-9abe-a81d687580af/main/0.log"
Sep 10 23:36:21.921935 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:21.921906    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-clkgr_294e8555-fe18-4f99-a48e-2228301d8875/main/0.log"
Sep 10 23:36:22.036451 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:22.036364    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9fllh_70a9812e-85d6-495f-99b5-630e413eb6fa/main/0.log"
Sep 10 23:36:30.232702 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:30.232668    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b44ws_70214487-51cd-4cf7-b0de-7425e698a6ec/global-pull-secret-syncer/0.log"
Sep 10 23:36:30.346605 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:30.346573    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fm9pt_67507844-de1a-4abb-b65c-950cae41f443/konnectivity-agent/0.log"
Sep 10 23:36:30.417246 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:30.417218    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-120.ec2.internal_4eeb794d498bff6249be66491c7495c4/haproxy/0.log"
Sep 10 23:36:34.950752 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:34.950717    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-c979f8d77-crf2j_5fc1ca76-9474-4173-a533-cfb06fb946c8/authorino/0.log"
Sep 10 23:36:34.995662 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:34.995633    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-9qvlr_394635a7-a40c-411e-83e3-81ac20c50416/manager/0.log"
Sep 10 23:36:35.146501 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:35.146420    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-595v2_8a0be99c-0a8d-4dd1-a720-4f57498d3c75/limitador/0.log"
Sep 10 23:36:36.843588 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.843560    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/alertmanager/0.log"
Sep 10 23:36:36.877709 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.877679    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/config-reloader/0.log"
Sep 10 23:36:36.908652 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.908624    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/kube-rbac-proxy-web/0.log"
Sep 10 23:36:36.945724 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.945693    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/kube-rbac-proxy/0.log"
Sep 10 23:36:36.972894 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.972867    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/kube-rbac-proxy-metric/0.log"
Sep 10 23:36:36.999590 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:36.999562    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/prom-label-proxy/0.log"
Sep 10 23:36:37.033181 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.033159    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5f858376-5b6b-4da1-b0b7-a19f271f4708/init-config-reloader/0.log"
Sep 10 23:36:37.074732 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.074703    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-nh5gm_21d6b7ee-8dba-4987-996d-38cd4cf1f37a/cluster-monitoring-operator/0.log"
Sep 10 23:36:37.191052 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.191027    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-59f4745cc9-zjvrn_59bdf5f2-de56-416e-b66c-64a1744fce59/metrics-server/0.log"
Sep 10 23:36:37.223010 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.222982    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-ch4xn_dd44ba6c-9080-47a6-b1db-39469c41a613/monitoring-plugin/0.log"
Sep 10 23:36:37.424831 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.424800    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6q2j_b23b5603-6431-4e7e-bbdf-e1c5246f4c7e/node-exporter/0.log"
Sep 10 23:36:37.446051 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.445985    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6q2j_b23b5603-6431-4e7e-bbdf-e1c5246f4c7e/kube-rbac-proxy/0.log"
Sep 10 23:36:37.470351 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.470333    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6q2j_b23b5603-6431-4e7e-bbdf-e1c5246f4c7e/init-textfile/0.log"
Sep 10 23:36:37.626714 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.626686    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/prometheus/0.log"
Sep 10 23:36:37.664505 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.664477    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/config-reloader/0.log"
Sep 10 23:36:37.691538 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.691512    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/thanos-sidecar/0.log"
Sep 10 23:36:37.715238 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.715220    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/kube-rbac-proxy-web/0.log"
Sep 10 23:36:37.744813 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.744789    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/kube-rbac-proxy/0.log"
Sep 10 23:36:37.788492 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.788466    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/kube-rbac-proxy-thanos/0.log"
Sep 10 23:36:37.825877 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.825848    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1a8a2642-a337-4148-81d5-f49696da1b2e/init-config-reloader/0.log"
Sep 10 23:36:37.919089 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.919060    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-hhmgm_4c8d9209-6d77-4212-acb7-85e58a28f0d4/prometheus-operator-admission-webhook/0.log"
Sep 10 23:36:37.958868 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.958789    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bfdd7fb6-97j55_70fbfabd-574a-489f-90b4-e9c3398f3d8c/telemeter-client/0.log"
Sep 10 23:36:37.984422 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:37.984386    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bfdd7fb6-97j55_70fbfabd-574a-489f-90b4-e9c3398f3d8c/reload/0.log"
Sep 10 23:36:38.009226 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.009204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-58bfdd7fb6-97j55_70fbfabd-574a-489f-90b4-e9c3398f3d8c/kube-rbac-proxy/0.log"
Sep 10 23:36:38.051515 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.051488    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/thanos-query/0.log"
Sep 10 23:36:38.077099 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.077076    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/kube-rbac-proxy-web/0.log"
Sep 10 23:36:38.099202 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.099178    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/kube-rbac-proxy/0.log"
Sep 10 23:36:38.123407 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.123382    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/prom-label-proxy/0.log"
Sep 10 23:36:38.146898 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.146878    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/kube-rbac-proxy-rules/0.log"
Sep 10 23:36:38.171393 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:38.171367    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8555fcfb8-m6dmm_1c3e48b1-48cd-454f-a128-14eb8486b892/kube-rbac-proxy-metrics/0.log"
Sep 10 23:36:40.640305 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.640260    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"]
Sep 10 23:36:40.640810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.640675    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="808d31c7-9099-47e8-918c-21587bddd77b" containerName="maas-api"
Sep 10 23:36:40.640810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.640692    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="808d31c7-9099-47e8-918c-21587bddd77b" containerName="maas-api"
Sep 10 23:36:40.640810 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.640766    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="808d31c7-9099-47e8-918c-21587bddd77b" containerName="maas-api"
Sep 10 23:36:40.644055 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.644037    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.646210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.646190    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r674h\"/\"openshift-service-ca.crt\""
Sep 10 23:36:40.646210 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.646206    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-r674h\"/\"kube-root-ca.crt\""
Sep 10 23:36:40.646870 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.646854    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-r674h\"/\"default-dockercfg-2cv5k\""
Sep 10 23:36:40.653071 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.653051    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"]
Sep 10 23:36:40.787175 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.787140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-sys\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.787384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.787191    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-podres\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.787384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.787236    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-lib-modules\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.787384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.787261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76tgc\" (UniqueName: \"kubernetes.io/projected/bc509158-8c15-4731-99ab-7bd50fc14c43-kube-api-access-76tgc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.787384 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.787355    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-proc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888073 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888037    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-sys\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-podres\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-lib-modules\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76tgc\" (UniqueName: \"kubernetes.io/projected/bc509158-8c15-4731-99ab-7bd50fc14c43-kube-api-access-76tgc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-proc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888175    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-sys\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888232    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-podres\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888267 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-lib-modules\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.888602 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.888277    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bc509158-8c15-4731-99ab-7bd50fc14c43-proc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.899077 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.899018    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76tgc\" (UniqueName: \"kubernetes.io/projected/bc509158-8c15-4731-99ab-7bd50fc14c43-kube-api-access-76tgc\") pod \"perf-node-gather-daemonset-t52pq\" (UID: \"bc509158-8c15-4731-99ab-7bd50fc14c43\") " pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:40.954353 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:40.954327    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:41.225260 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.225215    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"]
Sep 10 23:36:41.227963 ip-10-0-137-120 kubenswrapper[2565]: W0910 23:36:41.227932    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbc509158_8c15_4731_99ab_7bd50fc14c43.slice/crio-d971cd13516c3ee57a361496935dac9ca4985304b90cef78c184546a833a1e4d WatchSource:0}: Error finding container d971cd13516c3ee57a361496935dac9ca4985304b90cef78c184546a833a1e4d: Status 404 returned error can't find the container with id d971cd13516c3ee57a361496935dac9ca4985304b90cef78c184546a833a1e4d
Sep 10 23:36:41.632349 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.632272    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-w82t4_f1dc2ed7-1a8a-4b79-a3e7-3567b1ceb58d/volume-data-source-validator/0.log"
Sep 10 23:36:41.935921 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.935889    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq" event={"ID":"bc509158-8c15-4731-99ab-7bd50fc14c43","Type":"ContainerStarted","Data":"3831bafcf87ccea3f9dcbfab62b10a4410f721a46bef02f266d80ff012ae5b55"}
Sep 10 23:36:41.936348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.935954    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:41.936348 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.935977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq" event={"ID":"bc509158-8c15-4731-99ab-7bd50fc14c43","Type":"ContainerStarted","Data":"d971cd13516c3ee57a361496935dac9ca4985304b90cef78c184546a833a1e4d"}
Sep 10 23:36:41.955307 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:41.955246    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq" podStartSLOduration=1.955228934 podStartE2EDuration="1.955228934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:36:41.954093881 +0000 UTC m=+2324.240510828" watchObservedRunningTime="2026-09-10 23:36:41.955228934 +0000 UTC m=+2324.241645925"
Sep 10 23:36:42.729778 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:42.729744    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fsc8h_5c0bad51-63d6-45a8-9ce2-dab24b009d0d/dns/0.log"
Sep 10 23:36:42.755482 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:42.755453    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fsc8h_5c0bad51-63d6-45a8-9ce2-dab24b009d0d/kube-rbac-proxy/0.log"
Sep 10 23:36:42.884342 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:42.884317    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5d2lx_81390241-df7f-40c8-a3a2-25e9a1cdd7de/dns-node-resolver/0.log"
Sep 10 23:36:43.558370 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:43.558343    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6rvc4_8284fd76-acaf-4918-a0b3-cb7b06791f81/node-ca/0.log"
Sep 10 23:36:44.852406 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:44.852382    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9b555765-hzxqd_966b510e-6894-4f2a-ba50-d51034a47d7f/kube-auth-proxy/0.log"
Sep 10 23:36:45.881971 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:45.881939    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jnbw5_7a07f79f-2c5b-44dd-b7c9-556bfe289520/serve-healthcheck-canary/0.log"
Sep 10 23:36:46.527674 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:46.527646    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n5mf_4edd12a4-1332-449e-9c82-b9f8b63002a2/kube-rbac-proxy/0.log"
Sep 10 23:36:46.550625 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:46.550600    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n5mf_4edd12a4-1332-449e-9c82-b9f8b63002a2/exporter/0.log"
Sep 10 23:36:46.574495 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:46.574464    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n5mf_4edd12a4-1332-449e-9c82-b9f8b63002a2/extractor/0.log"
Sep 10 23:36:47.950921 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:47.950889    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-r674h/perf-node-gather-daemonset-t52pq"
Sep 10 23:36:49.424250 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:49.424218    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfbc5c4d-pc92l_f57bab69-4315-492d-b1e4-a99246d5999c/manager/0.log"
Sep 10 23:36:49.444106 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:49.444077    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfbc5c4d-pc92l_f57bab69-4315-492d-b1e4-a99246d5999c/copy-manifests/0.log"
Sep 10 23:36:51.098242 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:51.098204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67777b886f-hgmcj_995e3521-25d5-45b9-bbc2-3680d3182920/manager/0.log"
Sep 10 23:36:51.249334 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:51.249236    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9b555765-hzxqd_966b510e-6894-4f2a-ba50-d51034a47d7f/kube-auth-proxy/0.log"
Sep 10 23:36:53.115311 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:53.115215    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d9b555765-hzxqd_966b510e-6894-4f2a-ba50-d51034a47d7f/kube-auth-proxy/0.log"
Sep 10 23:36:56.330414 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:56.330384    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wz6k9_1965f0ec-a915-45e0-b3c4-830431034b52/migrator/0.log"
Sep 10 23:36:56.351230 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:56.351205    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-wz6k9_1965f0ec-a915-45e0-b3c4-830431034b52/graceful-termination/0.log"
Sep 10 23:36:57.797950 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.797916    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/kube-multus-additional-cni-plugins/0.log"
Sep 10 23:36:57.833521 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.833497    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/egress-router-binary-copy/0.log"
Sep 10 23:36:57.861671 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.861643    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/cni-plugins/0.log"
Sep 10 23:36:57.894365 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.894335    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/bond-cni-plugin/0.log"
Sep 10 23:36:57.921415 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.921381    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/routeoverride-cni/0.log"
Sep 10 23:36:57.945879 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.945854    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/whereabouts-cni-bincopy/0.log"
Sep 10 23:36:57.968656 ip-10-0-137-120 kubenswrapper[2565]: I0910 23:36:57.968634    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98vd8_841cb625-d4d4-4a37-917c-273376ce4128/whereabouts-cni/0.log"