Sep 10 17:36:23.007082 ip-10-0-140-165 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 17:36:23.411431 ip-10-0-140-165 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:23.411431 ip-10-0-140-165 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 17:36:23.411431 ip-10-0-140-165 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:23.411431 ip-10-0-140-165 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 17:36:23.411431 ip-10-0-140-165 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 17:36:23.413234 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.413166    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 17:36:23.417459 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417446    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.417459 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417460    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417463    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417473    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417477    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417480    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417483    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417485    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417488    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417491    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417494    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417496    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417499    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417501    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417504    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417507    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417509    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417512    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417515    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417519    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417523    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.417520 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417526    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417529    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417532    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417535    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417537    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417540    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417543    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417545    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417547    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417550    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417553    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417556    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417558    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417560    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417563    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417566    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417568    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417571    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417573    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417576    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.418033 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417578    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417581    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417583    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417586    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417588    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417591    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417593    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417595    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417598    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417601    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417603    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417607    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417610    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417613    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417617    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417620    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417623    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417626    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417628    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.418507 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417631    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417633    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417636    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417638    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417641    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417643    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417646    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417648    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417650    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417653    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417655    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417658    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417660    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417663    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417665    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417668    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417671    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417673    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417676    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417678    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.418987 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417681    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417683    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417687    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417690    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417692    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417694    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417697    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417700    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417702    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417705    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417708    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417712    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417715    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417717    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417720    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417722    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417725    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417727    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.417730    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.419466 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418389    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418395    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418398    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418401    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418404    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418407    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418411    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418415    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418418    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418422    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418425    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418428    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418431    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418433    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418436    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418438    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418441    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418444    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418446    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.419930 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418449    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418453    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418455    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418458    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418460    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418463    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418465    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418468    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418470    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418473    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418475    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418477    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418480    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418482    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418485    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418487    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418489    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418492    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418494    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418497    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.420415 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418500    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418503    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418506    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418510    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418512    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418515    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418517    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418520    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418523    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418525    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418529    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418533    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418535    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418538    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418541    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418543    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418545    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418548    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418550    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418553    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.420922 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418555    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418558    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418560    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418562    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418565    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418567    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418570    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418572    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418575    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418577    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418579    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418582    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418586    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418589    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418591    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418594    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418597    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418600    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418602    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418605    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.421384 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418607    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418609    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418614    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418617    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418620    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418623    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418626    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418629    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418632    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418634    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418637    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418639    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418642    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418644    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418647    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418649    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418651    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418654    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418656    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.421872 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.418659    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419017    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419022    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419025    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419027    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419031    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419034    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419036    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419039    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419042    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419045    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419048    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419050    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419053    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419056    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419060    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419063    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419066    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419069    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419072    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.422330 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419074    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419077    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419079    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419081    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419084    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419086    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419089    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419091    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419094    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419097    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419099    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419102    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419104    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419107    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419109    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419112    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419114    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419117    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419119    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419122    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.422843 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419124    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419127    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419132    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419134    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419137    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419139    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419141    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419144    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419147    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419149    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419152    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419154    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419157    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419159    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419162    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419164    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419166    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419169    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419171    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419173    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.423333 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419176    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419179    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419182    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419184    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419187    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419189    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419192    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419195    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419197    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419200    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419202    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419204    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419207    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419210    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419214    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419216    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419219    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419221    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419224    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.423830 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419226    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419229    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419231    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419234    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419236    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419239    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419241    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419246    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419249    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419252    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419255    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419258    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419260    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419263    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419266    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419269    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419273    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419275    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419279    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.424288 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419281    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419284    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419351    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419384    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419389    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419393    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419399    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419402    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419408    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419411    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419415    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419418    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419421    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419424    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419427    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419429    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419432    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419435    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419437    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419440    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419443    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419446    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419448    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419451    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 17:36:23.424753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419454    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419457    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419460    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419464    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419468    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419471    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419474    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419476    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419479    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419482    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419486    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419488    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419491    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419494    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419496    2571 flags.go:64] FLAG: --enable-server="true"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419499    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419504    2571 flags.go:64] FLAG: --event-burst="100"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419507    2571 flags.go:64] FLAG: --event-qps="50"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419510    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419513    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419516    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419519    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419522    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419524    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419527    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 10 17:36:23.425354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419530    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419532    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419535    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419538    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419540    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419543    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419546    2571 flags.go:64] FLAG: --feature-gates=""
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419549    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419552    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419554    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419558    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419561    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419564    2571 flags.go:64] FLAG: --help="false"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419567    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419569    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419572    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419575    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419578    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419581    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419584    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419586    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419589    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419591    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 17:36:23.426010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419595    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419598    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419601    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419604    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419607    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419609    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419612    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419615    2571 flags.go:64] FLAG: --lock-file=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419618    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419620    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419623    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419628    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419631    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419633    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419636    2571 flags.go:64] FLAG: --logging-format="text"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419639    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419642    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419644    2571 flags.go:64] FLAG: --manifest-url=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419647    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419652    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419655    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419659    2571 flags.go:64] FLAG: --max-pods="110"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419662    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419664    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419667    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 17:36:23.426619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419670    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419672    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419675    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419678    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419684    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419687    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419690    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419694    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419697    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419702    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419705    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419708    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419711    2571 flags.go:64] FLAG: --port="10250"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419714    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419717    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-041b13669fe98b5de"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419720    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419722    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419725    2571 flags.go:64] FLAG: --register-node="true"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419728    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419731    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419734    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419736    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419739    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419742    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419745    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 17:36:23.427241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419748    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419751    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419755    2571 flags.go:64] FLAG: --runonce="false"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419758    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419761    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419763    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419766    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419769    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419771    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419774    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419777    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419797    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419800    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419802    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419806    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419809    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419812    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419814    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419819    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419822    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419825    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419828    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419831    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419834    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419836    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 17:36:23.427872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419839    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419842    2571 flags.go:64] FLAG: --v="2"
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419845    2571 flags.go:64] FLAG: --version="false"
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419849    2571 flags.go:64] FLAG: --vmodule=""
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419852    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.419855    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419960    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419964    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419967    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419971    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419974    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419977    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419980    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419983    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419986    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419988    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419990    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419993    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419995    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.419998    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420002    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.428556 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420006    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420009    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420012    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420015    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420017    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420020    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420022    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420025    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420027    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420029    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420032    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420034    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420037    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420039    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420042    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420044    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420046    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420049    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420051    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420054    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.429084 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420058    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420060    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420063    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420066    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420068    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420071    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420073    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420076    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420079    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420081    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420084    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420086    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420089    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420091    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420093    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420096    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420098    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420101    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420103    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420105    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.429596 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420108    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420111    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420113    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420116    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420119    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420122    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420125    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420128    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420131    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420134    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420136    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420139    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420143    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420145    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420148    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420151    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420153    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420156    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420158    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.430109 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420161    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420163    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420167    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420170    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420173    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420175    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420178    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420180    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420182    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420185    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420188    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420190    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420192    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420195    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420201    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420203    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420206    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420208    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420211    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.430561 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420213    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420216    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420218    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420220    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420223    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.420225    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.420944    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.428832    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.428857    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.430052    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.430060    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430133    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430138    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430141    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430144    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.431045 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430147    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430150    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430153    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430155    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430158    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430160    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430163    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430165    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430168    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430171    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430175    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430179    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430182    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430184    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430187    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430190    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430192    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430195    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430198    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430201    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.431409 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430203    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430206    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430209    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430211    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430214    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430223    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430226    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430229    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430232    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430234    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430237    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430239    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430242    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430246    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430250    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430253    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430256    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430258    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430261    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.431911 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430263    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430266    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430269    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430271    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430274    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430276    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430279    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430281    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430285    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430288    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430291    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430294    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430297    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430300    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430303    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430306    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430308    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430311    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430313    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.432363 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430321    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430324    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430326    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430329    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430331    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430334    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430336    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430339    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430341    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430344    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430346    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430348    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430351    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430353    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430356    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430358    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430361    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430364    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430366    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430369    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.432920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430371    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430374    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430376    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430379    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430381    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430384    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430386    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430389    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430391    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430393    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430396    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430398    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430401    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430409    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430412    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430414    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.433402 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430416    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.430421    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430573    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430578    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430580    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430583    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430585    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430588    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430591    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430593    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430596    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430598    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430601    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430603    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430606    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430608    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 17:36:23.433811 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430610    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430613    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430615    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430618    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430620    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430623    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430625    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430628    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430630    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430633    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430635    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430637    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430640    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430642    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430650    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430653    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430655    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430658    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430660    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430663    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 17:36:23.434208 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430665    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430667    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430671    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430674    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430677    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430680    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430683    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430686    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430688    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430691    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430693    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430695    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430698    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430700    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430703    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430705    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430708    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430710    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430713    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 17:36:23.434937 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430715    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430718    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430720    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430722    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430725    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430728    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430731    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430734    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430743    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430746    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430748    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430751    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430753    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430755    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430758    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430760    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430763    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430765    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430768    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430770    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 17:36:23.435651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430772    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430775    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430778    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430798    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430801    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430804    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430806    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430809    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430812    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430814    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430817    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430819    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430822    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430824    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430826    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430829    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430831    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430833    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430836    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 17:36:23.436194 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430838    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430841    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430849    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430852    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430854    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430858    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:23.430861    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.430866    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.431005    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 17:36:23.436651 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.436350    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 17:36:23.437369 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.437358    2571 server.go:1019] "Starting client certificate rotation"
Sep 10 17:36:23.437472 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.437455    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:36:23.437500 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.437495    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 17:36:23.440076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.440063    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:36:23.442067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.442050    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 17:36:23.451155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.451140    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 10 17:36:23.455742 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.455725    2571 log.go:25] "Validated CRI v1 image API"
Sep 10 17:36:23.458668 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.458652    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 17:36:23.461700 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.461683    2571 fs.go:135] Filesystem UUIDs: map[0e8b801a-f46a-44e9-afe9-353982bd0888:/dev/nvme0n1p3 36273d74-8919-47c8-889a-8bde83fcff6b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 17:36:23.461757 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.461700    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 17:36:23.467357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.467251    2571 manager.go:216] Machine: {Timestamp:2026-09-10 17:36:23.465933395 +0000 UTC m=+0.341283813 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099031 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a8a72ada893d7473f4fb28abfb438 SystemUUID:ec2a8a72-ada8-93d7-473f-4fb28abfb438 BootID:1ffdefc6-712b-4df8-854c-5a52cbc1603a Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d0:68:01:20:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d0:68:01:20:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:11:e1:4d:db:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 17:36:23.467357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.467348    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 17:36:23.467475 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.467443    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 17:36:23.468593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.468576    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:36:23.469170 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.469145    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 17:36:23.469308 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.469173    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 17:36:23.469982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.469973    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 17:36:23.470013 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.469986    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 17:36:23.470013 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.469993    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:36:23.470822 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.470811    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 17:36:23.472161 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.472151    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 17:36:23.476708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476697    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:36:23.476830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476820    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 17:36:23.476859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476848    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 17:36:23.476892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476864    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 17:36:23.476892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476883    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 17:36:23.476946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476895    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 10 17:36:23.476946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.476903    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 17:36:23.477438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.477420    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:36:23.477514 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.477456    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 17:36:23.480536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.480509    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 17:36:23.483281 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.483264    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 17:36:23.483355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.483303    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 17:36:23.484094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484082    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484103    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484113    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484122    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484133    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484141    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484161    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 17:36:23.484171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484169    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 17:36:23.484377 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484179    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 17:36:23.484377 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484187    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 17:36:23.484377 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484199    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 17:36:23.484377 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484208    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 17:36:23.484959 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484948    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 17:36:23.485021 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.484961    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 17:36:23.488004 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.487960    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 17:36:23.488004 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.487994    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 17:36:23.488133 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.488010    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 17:36:23.488641 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.488627    2571 server.go:1268] "Started kubelet"
Sep 10 17:36:23.488752 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.488704    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 17:36:23.488860 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.488715    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 17:36:23.488860 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.488806    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 17:36:23.489413 ip-10-0-140-165 systemd[1]: Started Kubernetes Kubelet.
Sep 10 17:36:23.490419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.490401    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 17:36:23.490873 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.490858    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 17:36:23.491918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.491900    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xdvvr"
Sep 10 17:36:23.494475 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.494452    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 17:36:23.495482 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.495459    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 17:36:23.496126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.496112    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 17:36:23.496272 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.496243    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.496838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.496822    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 17:36:23.496838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.496840    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 17:36:23.496985 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.496824    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 17:36:23.497035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.496991    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 17:36:23.497035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497000    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 17:36:23.497411 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497397    2571 factory.go:55] Registering systemd factory
Sep 10 17:36:23.497454 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497413    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 10 17:36:23.497598 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497585    2571 factory.go:153] Registering CRI-O factory
Sep 10 17:36:23.497653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497599    2571 factory.go:223] Registration of the crio container factory successfully
Sep 10 17:36:23.497653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497640    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 17:36:23.497653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497652    2571 factory.go:103] Registering Raw factory
Sep 10 17:36:23.497763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.497662    2571 manager.go:1196] Started watching for new ooms in manager
Sep 10 17:36:23.498036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.498023    2571 manager.go:349] Starting recovery of all containers
Sep 10 17:36:23.499497 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.499478    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xdvvr"
Sep 10 17:36:23.501759 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.501731    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 17:36:23.502364 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.502338    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 17:36:23.504750 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.502406    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-165.ec2.internal.18d4063696109d3e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-165.ec2.internal,UID:ip-10-0-140-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-165.ec2.internal,},FirstTimestamp:2026-09-10 17:36:23.488601406 +0000 UTC m=+0.363951824,LastTimestamp:2026-09-10 17:36:23.488601406 +0000 UTC m=+0.363951824,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-165.ec2.internal,}"
Sep 10 17:36:23.509507 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.509491    2571 manager.go:354] Recovery completed
Sep 10 17:36:23.513613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.513601    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.515605 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.515587    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.515675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.515620    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.515675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.515631    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.516166 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516149    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 17:36:23.516166 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516160    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.516166 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516166    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 17:36:23.516324 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516181    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 17:36:23.516859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516774    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.516948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516886    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.516948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.516904    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.517503 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.517490    2571 policy_none.go:49] "None policy: Start"
Sep 10 17:36:23.517555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.517506    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 17:36:23.517555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.517515    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 17:36:23.518663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.518648    2571 policy_none.go:47] "Start"
Sep 10 17:36:23.555404 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555388    2571 manager.go:337] "Starting Device Plugin manager"
Sep 10 17:36:23.555495 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.555428    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 17:36:23.555495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555439    2571 server.go:87] "Starting device plugin registration server"
Sep 10 17:36:23.555658 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555644    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 17:36:23.555721 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555659    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 17:36:23.555809 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555778    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 17:36:23.555878 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555866    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 17:36:23.555934 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.555880    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 17:36:23.557238 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.556373    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 17:36:23.557238 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.556498    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.641994 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.641957    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 17:36:23.643235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.643219    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 17:36:23.643306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.643272    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 17:36:23.643306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.643298    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 17:36:23.643410 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.643341    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 17:36:23.645896 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.645872    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:23.656093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.656074    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.656776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.656763    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.656849 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.656808    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.656849 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.656820    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.656849 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.656839    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.666028 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.665996    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.666028 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.666015    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-165.ec2.internal\": node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.680029 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.680011    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.743636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.743615    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"]
Sep 10 17:36:23.743707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.743681    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.745155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.745140    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.745210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.745168    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.745210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.745182    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.746392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.746381    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.746497 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.746485    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.746532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.746516    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.747084 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747061    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.747168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747091    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.747168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747102    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.747168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747062    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.747268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747169    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.747268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.747180    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.748511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.748498    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.748580 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.748520    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 17:36:23.749133 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.749117    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 17:36:23.749214 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.749150    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 17:36:23.749214 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.749168    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeHasSufficientPID"
Sep 10 17:36:23.778334 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.778317    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-165.ec2.internal\" not found" node="ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.780075 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.780049    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.781761 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.781745    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-165.ec2.internal\" not found" node="ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.798034 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.798013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.798120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.798037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.798120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.798053    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6fbc923c8aeb003a0cb6b765d09dd22b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"6fbc923c8aeb003a0cb6b765d09dd22b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.880863 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.880842    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:23.898156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.898156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898145    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.898278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.898278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6fbc923c8aeb003a0cb6b765d09dd22b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"6fbc923c8aeb003a0cb6b765d09dd22b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.898278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898249    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6fbc923c8aeb003a0cb6b765d09dd22b-config\") pod \"kube-apiserver-proxy-ip-10-0-140-165.ec2.internal\" (UID: \"6fbc923c8aeb003a0cb6b765d09dd22b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.898278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:23.898264    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de8ddf2b88624abfe9e09be17393198c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal\" (UID: \"de8ddf2b88624abfe9e09be17393198c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:23.981578 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:23.981520    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:24.082067 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.082030    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:24.082067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.082063    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:24.084547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.084516    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:24.183125 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.183097    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:24.283619 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.283570    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-165.ec2.internal\" not found"
Sep 10 17:36:24.326025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.325999    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:24.396763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.396738    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:24.405664 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.405640    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:36:24.407871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.407855    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal"
Sep 10 17:36:24.417368 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.417347    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 17:36:24.437510 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.437487    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 17:36:24.437660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.437637    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 17:36:24.437701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.437647    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 17:36:24.477816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.477776    2571 apiserver.go:52] "Watching apiserver"
Sep 10 17:36:24.495676 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.495647    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 17:36:24.495976 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.495957    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 17:36:24.496380 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.496354    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-pj2gf","openshift-network-operator/iptables-alerter-5v9gh","openshift-ovn-kubernetes/ovnkube-node-cvl9w","kube-system/global-pull-secret-syncer-79kh2","kube-system/konnectivity-agent-4szqx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt","openshift-cluster-node-tuning-operator/tuned-wpmlg","openshift-dns/node-resolver-pttp4","openshift-image-registry/node-ca-crqvl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal","openshift-multus/multus-additional-cni-plugins-t8l7c","kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal","openshift-multus/multus-rtggj","openshift-multus/network-metrics-daemon-czh2q"]
Sep 10 17:36:24.498241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.498226    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:24.498308 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.498289    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:24.499339 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.499324    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.500707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.500690    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.501087 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501065    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:24.501707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501560    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9rq7\""
Sep 10 17:36:24.501707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501583    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 17:36:24.501707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501605    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 17:31:24 +0000 UTC" deadline="2028-05-14 23:20:56.827185699 +0000 UTC"
Sep 10 17:36:24.501707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501641    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14693h44m32.325546951s"
Sep 10 17:36:24.501707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501586    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.502035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.501729    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.502080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.502067    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.502156 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.502134    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:24.502949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.502908    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.503058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.502989    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 17:36:24.503530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.503516    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.504195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.503975    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 17:36:24.504195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.503975    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 17:36:24.504195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.504020    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.504195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.504080    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2t96x\""
Sep 10 17:36:24.504391 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.504371    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 17:36:24.505062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.505042    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.505675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.505656    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 17:36:24.505774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.505739    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-glkpd\""
Sep 10 17:36:24.506083 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.506060    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 17:36:24.506628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.506614    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.507154 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.507135    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 17:36:24.507238 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.507199    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mcxr9\""
Sep 10 17:36:24.507308 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.507270    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.507369 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.507320    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.507508 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.507492    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 17:36:24.508223 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.508207    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.508697 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.508680    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fhwhr\""
Sep 10 17:36:24.509018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.509001    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.509099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.509030    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.510014 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.509994    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.510456 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.510436    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.510563 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.510543    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j2qvw\""
Sep 10 17:36:24.510822 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.510804    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.511758 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.511691    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.512360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.512254    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 17:36:24.512360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.512269    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.512506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.512486    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.512601 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.512522    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r6cnt\""
Sep 10 17:36:24.513293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.513279    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.513850 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.513833    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 17:36:24.513902 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.513856    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 17:36:24.514004 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.513958    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 17:36:24.514227 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.514168    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 17:36:24.514227 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.514177    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 17:36:24.514349 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.514267    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vhqr9\""
Sep 10 17:36:24.514999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.514980    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.515069 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.515052    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:24.515401 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.515386    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 17:36:24.515463 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.515425    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fh95l\""
Sep 10 17:36:24.531293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.531276    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dzng4"
Sep 10 17:36:24.540867 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.540837    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:24.551308 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.551291    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dzng4"
Sep 10 17:36:24.598366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.598349    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 17:36:24.601893 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601875    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-kubernetes\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.601964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601898    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-tmp\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.601964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601916    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-multus\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.601964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601931    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e5026e-a37f-45a4-a898-89f2c1052576-host\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.602056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-ovn\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.601986    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-config\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.602056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-run\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-netns\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-kubelet\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602125    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-slash\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602156    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovn-node-metrics-cert\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-conf-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-host\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602234    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgv7t\" (UniqueName: \"kubernetes.io/projected/26515fca-4074-4d40-b42c-235ffebf01c3-kube-api-access-pgv7t\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm74h\" (UniqueName: \"kubernetes.io/projected/b135c9cb-01d1-4ac5-a31f-720cc697b74b-kube-api-access-xm74h\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a79445c3-db0e-4891-832e-833f2c087675-host-slash\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602315    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-script-lib\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602329    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-socket-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602351    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-registration-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-sys\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602393    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-var-lib-kubelet\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26515fca-4074-4d40-b42c-235ffebf01c3-hosts-file\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-kubelet\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-system-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602477    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-multus-daemon-config\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602510    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k865v\" (UniqueName: \"kubernetes.io/projected/6adb3bcb-c09a-4291-8509-84188419571e-kube-api-access-k865v\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602528    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-kubelet-config\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:24.602663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sdm4\" (UniqueName: \"kubernetes.io/projected/54b70f12-c630-4c43-863e-7dcfd6f0c68d-kube-api-access-7sdm4\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602687    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6bz5\" (UniqueName: \"kubernetes.io/projected/cd4bebfe-d416-4305-abe5-3a401fa91ac8-kube-api-access-c6bz5\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602723    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cnibin\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf65b\" (UniqueName: \"kubernetes.io/projected/a7e5026e-a37f-45a4-a898-89f2c1052576-kube-api-access-rf65b\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602766    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-etc-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602799    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-tuned\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-dbus\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v69qb\" (UniqueName: \"kubernetes.io/projected/22631be7-f152-457e-a647-4dda83f32fa9-kube-api-access-v69qb\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602852    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a79445c3-db0e-4891-832e-833f2c087675-iptables-alerter-script\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602873    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-systemd-units\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602888    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-systemd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602903    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-cnibin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-node-log\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysconfig\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.602984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602984    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-cni-binary-copy\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.602999    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7e5026e-a37f-45a4-a898-89f2c1052576-serviceca\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603058    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-netns\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603085    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-env-overrides\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603106    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-systemd\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603122    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-bin\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603149    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-sys-fs\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-conf\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603185    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-lib-modules\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603199    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-netd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-device-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.603524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603263    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-modprobe-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603286    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26515fca-4074-4d40-b42c-235ffebf01c3-tmp-dir\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603315    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603339    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-bin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603358    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-multus-certs\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-log-socket\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-system-cni-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-socket-dir-parent\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603452    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-k8s-cni-cncf-io\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603466    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-hostroot\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603489    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-etc-kubernetes\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603507    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5baaa61-1d31-4de2-b8bb-42cffda54965-agent-certs\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603521    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5baaa61-1d31-4de2-b8bb-42cffda54965-konnectivity-ca\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdnx6\" (UniqueName: \"kubernetes.io/projected/a79445c3-db0e-4891-832e-833f2c087675-kube-api-access-sdnx6\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-var-lib-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.604046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.603591    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgml8\" (UniqueName: \"kubernetes.io/projected/eb597048-2bb8-4d1d-9e89-40b63492abc4-kube-api-access-lgml8\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.608590 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.608565    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:24.608590 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.608582    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:24.608590 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.608591    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:24.608759 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.608636    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:25.108622479 +0000 UTC m=+1.983972888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:24.648457 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.648441    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:24.664149 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.664122    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbc923c8aeb003a0cb6b765d09dd22b.slice/crio-46affc94589233cfaf345910f308a2d457b01968862af9d11a8ca84af464e9f4 WatchSource:0}: Error finding container 46affc94589233cfaf345910f308a2d457b01968862af9d11a8ca84af464e9f4: Status 404 returned error can't find the container with id 46affc94589233cfaf345910f308a2d457b01968862af9d11a8ca84af464e9f4
Sep 10 17:36:24.664311 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.664294    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8ddf2b88624abfe9e09be17393198c.slice/crio-e7e7ac0110a7cfb5c2a38f18a4c823c7bd6336bfcc74c6bef5449675da21abe7 WatchSource:0}: Error finding container e7e7ac0110a7cfb5c2a38f18a4c823c7bd6336bfcc74c6bef5449675da21abe7: Status 404 returned error can't find the container with id e7e7ac0110a7cfb5c2a38f18a4c823c7bd6336bfcc74c6bef5449675da21abe7
Sep 10 17:36:24.669026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.669013    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:36:24.704754 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704721    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5baaa61-1d31-4de2-b8bb-42cffda54965-agent-certs\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.704864 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5baaa61-1d31-4de2-b8bb-42cffda54965-konnectivity-ca\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.704864 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704775    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdnx6\" (UniqueName: \"kubernetes.io/projected/a79445c3-db0e-4891-832e-833f2c087675-kube-api-access-sdnx6\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.704864 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704820    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-var-lib-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704870    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-var-lib-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgml8\" (UniqueName: \"kubernetes.io/projected/eb597048-2bb8-4d1d-9e89-40b63492abc4-kube-api-access-lgml8\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.705006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.704958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-kubernetes\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705102 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705034    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-kubernetes\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705102 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705065    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-tmp\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705102 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705094    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-multus\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e5026e-a37f-45a4-a898-89f2c1052576-host\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-ovn\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705130    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-multus\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705169    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e5026e-a37f-45a4-a898-89f2c1052576-host\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705190    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-config\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705189    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-ovn\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-run\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-netns\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-kubelet\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705299    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-run\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-kubelet\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705315    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-netns\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-slash\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705383    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovn-node-metrics-cert\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705392    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-slash\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705402    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705405    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-conf-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705412    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705434    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705456    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-host\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgv7t\" (UniqueName: \"kubernetes.io/projected/26515fca-4074-4d40-b42c-235ffebf01c3-kube-api-access-pgv7t\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705517    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-conf-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.705611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705551    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm74h\" (UniqueName: \"kubernetes.io/projected/b135c9cb-01d1-4ac5-a31f-720cc697b74b-kube-api-access-xm74h\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705568    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-host\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a79445c3-db0e-4891-832e-833f2c087675-host-slash\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-script-lib\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a79445c3-db0e-4891-832e-833f2c087675-host-slash\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-socket-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705744    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-config\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705830    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-registration-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705884    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-registration-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705902    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-socket-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705912    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-sys\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5baaa61-1d31-4de2-b8bb-42cffda54965-konnectivity-ca\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-var-lib-kubelet\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705956    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-sys\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.705989    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26515fca-4074-4d40-b42c-235ffebf01c3-hosts-file\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.706413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706025    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-var-lib-kubelet\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-kubelet\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706083    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26515fca-4074-4d40-b42c-235ffebf01c3-hosts-file\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706142    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-kubelet\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovnkube-script-lib\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706178    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706216    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-system-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706222    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-multus-daemon-config\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706267    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-system-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k865v\" (UniqueName: \"kubernetes.io/projected/6adb3bcb-c09a-4291-8509-84188419571e-kube-api-access-k865v\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-kubelet-config\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706398    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-kubelet-config\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.706492    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.706565    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:25.206547562 +0000 UTC m=+2.081897975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:24.707428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sdm4\" (UniqueName: \"kubernetes.io/projected/54b70f12-c630-4c43-863e-7dcfd6f0c68d-kube-api-access-7sdm4\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6bz5\" (UniqueName: \"kubernetes.io/projected/cd4bebfe-d416-4305-abe5-3a401fa91ac8-kube-api-access-c6bz5\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cnibin\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706662    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf65b\" (UniqueName: \"kubernetes.io/projected/a7e5026e-a37f-45a4-a898-89f2c1052576-kube-api-access-rf65b\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-etc-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706701    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-multus-daemon-config\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706707    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-tuned\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706870    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cnibin\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706939    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-dbus\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.706966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v69qb\" (UniqueName: \"kubernetes.io/projected/22631be7-f152-457e-a647-4dda83f32fa9-kube-api-access-v69qb\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a79445c3-db0e-4891-832e-833f2c087675-iptables-alerter-script\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-systemd-units\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707039    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-etc-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707050    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-systemd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-cnibin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707092    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-node-log\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b00454b7-c84c-4280-87ca-39149012189f-dbus\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.708256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysconfig\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707136    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707159    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-cni-binary-copy\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707170    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-node-log\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707202    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7e5026e-a37f-45a4-a898-89f2c1052576-serviceca\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-netns\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707279    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-env-overrides\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707247    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-cnibin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707430    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-systemd\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysconfig\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707309    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-systemd\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-bin\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-sys-fs\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707591    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-conf\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-lib-modules\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.709772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707640    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-netd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707687    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-device-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707742    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-modprobe-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707742    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a79445c3-db0e-4891-832e-833f2c087675-iptables-alerter-script\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707759    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707769    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26515fca-4074-4d40-b42c-235ffebf01c3-tmp-dir\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-systemd-units\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-bin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-multus-certs\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707954    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-log-socket\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-systemd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.707979    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-system-cni-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-socket-dir-parent\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-system-cni-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.710288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708038    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-k8s-cni-cncf-io\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708079    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-k8s-cni-cncf-io\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-socket-dir-parent\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708164    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-run-multus-certs\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-host-var-lib-cni-bin\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708228    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6adb3bcb-c09a-4291-8509-84188419571e-cni-binary-copy\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-run-netns\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708516    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b135c9cb-01d1-4ac5-a31f-720cc697b74b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-hostroot\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708594    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-etc-kubernetes\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708642    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26515fca-4074-4d40-b42c-235ffebf01c3-tmp-dir\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b70f12-c630-4c43-863e-7dcfd6f0c68d-env-overrides\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708665    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-etc-kubernetes\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-hostroot\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708727    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-device-dir\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708812    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-lib-modules\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.708824    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708855    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-bin\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.710743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708812    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b135c9cb-01d1-4ac5-a31f-720cc697b74b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708858    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-sys-fs\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:24.708879    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:25.208862532 +0000 UTC m=+2.084212938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708921    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-host-cni-netd\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708936    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eb597048-2bb8-4d1d-9e89-40b63492abc4-etc-selinux\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708958    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-sysctl-conf\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.708977    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-run-openvswitch\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709004    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-modprobe-d\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6adb3bcb-c09a-4291-8509-84188419571e-multus-cni-dir\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709069    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b70f12-c630-4c43-863e-7dcfd6f0c68d-log-socket\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709171    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-etc-tuned\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709194    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a7e5026e-a37f-45a4-a898-89f2c1052576-serviceca\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cd4bebfe-d416-4305-abe5-3a401fa91ac8-tmp\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b70f12-c630-4c43-863e-7dcfd6f0c68d-ovn-node-metrics-cert\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.711315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.709509    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5baaa61-1d31-4de2-b8bb-42cffda54965-agent-certs\") pod \"konnectivity-agent-4szqx\" (UID: \"e5baaa61-1d31-4de2-b8bb-42cffda54965\") " pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.717322 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.717297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6bz5\" (UniqueName: \"kubernetes.io/projected/cd4bebfe-d416-4305-abe5-3a401fa91ac8-kube-api-access-c6bz5\") pod \"tuned-wpmlg\" (UID: \"cd4bebfe-d416-4305-abe5-3a401fa91ac8\") " pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.717580 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.717555    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgv7t\" (UniqueName: \"kubernetes.io/projected/26515fca-4074-4d40-b42c-235ffebf01c3-kube-api-access-pgv7t\") pod \"node-resolver-pttp4\" (UID: \"26515fca-4074-4d40-b42c-235ffebf01c3\") " pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.718202 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v69qb\" (UniqueName: \"kubernetes.io/projected/22631be7-f152-457e-a647-4dda83f32fa9-kube-api-access-v69qb\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:24.718293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718275    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sdm4\" (UniqueName: \"kubernetes.io/projected/54b70f12-c630-4c43-863e-7dcfd6f0c68d-kube-api-access-7sdm4\") pod \"ovnkube-node-cvl9w\" (UID: \"54b70f12-c630-4c43-863e-7dcfd6f0c68d\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.718394 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718371    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdnx6\" (UniqueName: \"kubernetes.io/projected/a79445c3-db0e-4891-832e-833f2c087675-kube-api-access-sdnx6\") pod \"iptables-alerter-5v9gh\" (UID: \"a79445c3-db0e-4891-832e-833f2c087675\") " pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.718620 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718606    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf65b\" (UniqueName: \"kubernetes.io/projected/a7e5026e-a37f-45a4-a898-89f2c1052576-kube-api-access-rf65b\") pod \"node-ca-crqvl\" (UID: \"a7e5026e-a37f-45a4-a898-89f2c1052576\") " pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.718668 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718636    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k865v\" (UniqueName: \"kubernetes.io/projected/6adb3bcb-c09a-4291-8509-84188419571e-kube-api-access-k865v\") pod \"multus-rtggj\" (UID: \"6adb3bcb-c09a-4291-8509-84188419571e\") " pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.718970 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.718955    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm74h\" (UniqueName: \"kubernetes.io/projected/b135c9cb-01d1-4ac5-a31f-720cc697b74b-kube-api-access-xm74h\") pod \"multus-additional-cni-plugins-t8l7c\" (UID: \"b135c9cb-01d1-4ac5-a31f-720cc697b74b\") " pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.719079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.719063    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgml8\" (UniqueName: \"kubernetes.io/projected/eb597048-2bb8-4d1d-9e89-40b63492abc4-kube-api-access-lgml8\") pod \"aws-ebs-csi-driver-node-htptt\" (UID: \"eb597048-2bb8-4d1d-9e89-40b63492abc4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.816342 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.816296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5v9gh"
Sep 10 17:36:24.824702 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.824683    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79445c3_db0e_4891_832e_833f2c087675.slice/crio-250faded8f3a5dc401303acbb555de84f895db159b008a70be6880d9c71d1290 WatchSource:0}: Error finding container 250faded8f3a5dc401303acbb555de84f895db159b008a70be6880d9c71d1290: Status 404 returned error can't find the container with id 250faded8f3a5dc401303acbb555de84f895db159b008a70be6880d9c71d1290
Sep 10 17:36:24.835920 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.835905    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:24.842889 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.842866    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b70f12_c630_4c43_863e_7dcfd6f0c68d.slice/crio-0dd391ac9e578c79509a7f7c9d4d70b772ece1060353f68f2f6e7b79ac88da40 WatchSource:0}: Error finding container 0dd391ac9e578c79509a7f7c9d4d70b772ece1060353f68f2f6e7b79ac88da40: Status 404 returned error can't find the container with id 0dd391ac9e578c79509a7f7c9d4d70b772ece1060353f68f2f6e7b79ac88da40
Sep 10 17:36:24.859430 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.859411    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:24.866159 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.866139    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5baaa61_1d31_4de2_b8bb_42cffda54965.slice/crio-93f018462c786cbb181e259133d2fd0aa373044faa83acbadee9cc0e0bc57a2b WatchSource:0}: Error finding container 93f018462c786cbb181e259133d2fd0aa373044faa83acbadee9cc0e0bc57a2b: Status 404 returned error can't find the container with id 93f018462c786cbb181e259133d2fd0aa373044faa83acbadee9cc0e0bc57a2b
Sep 10 17:36:24.879035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.879020    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt"
Sep 10 17:36:24.884833 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.884816    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wpmlg"
Sep 10 17:36:24.885456 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.885436    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb597048_2bb8_4d1d_9e89_40b63492abc4.slice/crio-2d6d34494674cefcce181d9f518a4c3c7c7284848b3fcfa9e209e58acd4faae7 WatchSource:0}: Error finding container 2d6d34494674cefcce181d9f518a4c3c7c7284848b3fcfa9e209e58acd4faae7: Status 404 returned error can't find the container with id 2d6d34494674cefcce181d9f518a4c3c7c7284848b3fcfa9e209e58acd4faae7
Sep 10 17:36:24.891101 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.891080    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4bebfe_d416_4305_abe5_3a401fa91ac8.slice/crio-37f9288f5602f1a353c55a5910fa12c6f46a60297c79bdd0216f00dc554444f6 WatchSource:0}: Error finding container 37f9288f5602f1a353c55a5910fa12c6f46a60297c79bdd0216f00dc554444f6: Status 404 returned error can't find the container with id 37f9288f5602f1a353c55a5910fa12c6f46a60297c79bdd0216f00dc554444f6
Sep 10 17:36:24.898366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.898351    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pttp4"
Sep 10 17:36:24.905108 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.905090    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26515fca_4074_4d40_b42c_235ffebf01c3.slice/crio-2eb35509ebf1cc5553f58085e44d70d1b09b34f9650836459f840a1ae24f77ba WatchSource:0}: Error finding container 2eb35509ebf1cc5553f58085e44d70d1b09b34f9650836459f840a1ae24f77ba: Status 404 returned error can't find the container with id 2eb35509ebf1cc5553f58085e44d70d1b09b34f9650836459f840a1ae24f77ba
Sep 10 17:36:24.909664 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.909651    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-crqvl"
Sep 10 17:36:24.916594 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.916579    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e5026e_a37f_45a4_a898_89f2c1052576.slice/crio-6435b7fd72b2a1016b05f8055e1739b1dce901c619b10b1d461e28239cdf22f4 WatchSource:0}: Error finding container 6435b7fd72b2a1016b05f8055e1739b1dce901c619b10b1d461e28239cdf22f4: Status 404 returned error can't find the container with id 6435b7fd72b2a1016b05f8055e1739b1dce901c619b10b1d461e28239cdf22f4
Sep 10 17:36:24.922389 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.922373    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8l7c"
Sep 10 17:36:24.926357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:24.926338    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtggj"
Sep 10 17:36:24.929386 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.929368    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb135c9cb_01d1_4ac5_a31f_720cc697b74b.slice/crio-86429bc83a0d37a8fd4ed9fb8f9c5c2e1c9b3bfe27b3e3ca7b81b1fa920a2e99 WatchSource:0}: Error finding container 86429bc83a0d37a8fd4ed9fb8f9c5c2e1c9b3bfe27b3e3ca7b81b1fa920a2e99: Status 404 returned error can't find the container with id 86429bc83a0d37a8fd4ed9fb8f9c5c2e1c9b3bfe27b3e3ca7b81b1fa920a2e99
Sep 10 17:36:24.933453 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:36:24.933435    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adb3bcb_c09a_4291_8509_84188419571e.slice/crio-10a21522864611fc396c920af42add50ad5877dc834d284705f4058c07c9baf8 WatchSource:0}: Error finding container 10a21522864611fc396c920af42add50ad5877dc834d284705f4058c07c9baf8: Status 404 returned error can't find the container with id 10a21522864611fc396c920af42add50ad5877dc834d284705f4058c07c9baf8
Sep 10 17:36:26.468985 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:26.468922    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:26.469105 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.469076    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:26.469105 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.469095    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:26.469214 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.469107    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:26.469214 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.469171    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:27.469145758 +0000 UTC m=+2.986825901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:26.569865 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:26.569837    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:26.569996 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:26.569901    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:26.570049 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.570005    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:26.570049 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.570013    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:26.570121 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.570078    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:27.570059529 +0000 UTC m=+3.087739675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:26.570121 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:26.570095    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:27.570085672 +0000 UTC m=+3.087765814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:26.911736 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:26.911651    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:31:25 +0000 UTC" deadline="2028-05-04 00:20:19.122916346 +0000 UTC"
Sep 10 17:36:26.911736 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:26.911689    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14430h43m52.211232287s"
Sep 10 17:36:27.041325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.041258    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtggj" event={"ID":"6adb3bcb-c09a-4291-8509-84188419571e","Type":"ContainerStarted","Data":"10a21522864611fc396c920af42add50ad5877dc834d284705f4058c07c9baf8"}
Sep 10 17:36:27.050119 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.049120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerStarted","Data":"86429bc83a0d37a8fd4ed9fb8f9c5c2e1c9b3bfe27b3e3ca7b81b1fa920a2e99"}
Sep 10 17:36:27.060964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.060911    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crqvl" event={"ID":"a7e5026e-a37f-45a4-a898-89f2c1052576","Type":"ContainerStarted","Data":"6435b7fd72b2a1016b05f8055e1739b1dce901c619b10b1d461e28239cdf22f4"}
Sep 10 17:36:27.081095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.080890    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt" event={"ID":"eb597048-2bb8-4d1d-9e89-40b63492abc4","Type":"ContainerStarted","Data":"2d6d34494674cefcce181d9f518a4c3c7c7284848b3fcfa9e209e58acd4faae7"}
Sep 10 17:36:27.101269 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.101195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4szqx" event={"ID":"e5baaa61-1d31-4de2-b8bb-42cffda54965","Type":"ContainerStarted","Data":"93f018462c786cbb181e259133d2fd0aa373044faa83acbadee9cc0e0bc57a2b"}
Sep 10 17:36:27.127228 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.127198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"0dd391ac9e578c79509a7f7c9d4d70b772ece1060353f68f2f6e7b79ac88da40"}
Sep 10 17:36:27.136060 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.136032    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5v9gh" event={"ID":"a79445c3-db0e-4891-832e-833f2c087675","Type":"ContainerStarted","Data":"250faded8f3a5dc401303acbb555de84f895db159b008a70be6880d9c71d1290"}
Sep 10 17:36:27.154610 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.154577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"de8ddf2b88624abfe9e09be17393198c","Type":"ContainerStarted","Data":"e7e7ac0110a7cfb5c2a38f18a4c823c7bd6336bfcc74c6bef5449675da21abe7"}
Sep 10 17:36:27.157212 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.157189    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pttp4" event={"ID":"26515fca-4074-4d40-b42c-235ffebf01c3","Type":"ContainerStarted","Data":"2eb35509ebf1cc5553f58085e44d70d1b09b34f9650836459f840a1ae24f77ba"}
Sep 10 17:36:27.168151 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.168068    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpmlg" event={"ID":"cd4bebfe-d416-4305-abe5-3a401fa91ac8","Type":"ContainerStarted","Data":"37f9288f5602f1a353c55a5910fa12c6f46a60297c79bdd0216f00dc554444f6"}
Sep 10 17:36:27.178285 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.178219    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" event={"ID":"6fbc923c8aeb003a0cb6b765d09dd22b","Type":"ContainerStarted","Data":"46affc94589233cfaf345910f308a2d457b01968862af9d11a8ca84af464e9f4"}
Sep 10 17:36:27.268544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.268516    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:27.475976 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.475893    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:27.476140 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.476052    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:27.476140 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.476069    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:27.476140 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.476081    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:27.476140 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.476137    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:29.476119228 +0000 UTC m=+4.993799370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:27.576236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.576199    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:27.576401 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.576265    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:27.576401 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.576372    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:27.576541 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.576468    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:29.576431103 +0000 UTC m=+5.094111247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:27.576838 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.576377    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:27.576912 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:27.576876    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:29.576859459 +0000 UTC m=+5.094539602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:27.912776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.912605    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 17:31:25 +0000 UTC" deadline="2028-05-28 07:19:13.815741376 +0000 UTC"
Sep 10 17:36:27.912776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:27.912660    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15013h42m45.903085371s"
Sep 10 17:36:28.001823 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:28.001795    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:28.001977 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:28.001920    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:28.002341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:28.002322    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:28.002444 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:28.002424    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:28.002551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:28.002519    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:28.002623 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:28.002605    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:28.897816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:28.897628    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 17:36:29.497828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:29.497793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:29.498276 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.497967    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:29.498276 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.497989    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:29.498276 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.498001    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:29.498276 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.498059    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:33.498040313 +0000 UTC m=+9.015720465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:29.598963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:29.598931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:29.599126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:29.598999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:29.599126 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.599074    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:29.599126 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.599108    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:29.599126 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.599128    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:33.599113465 +0000 UTC m=+9.116793609 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:29.599349 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:29.599167    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:33.599141153 +0000 UTC m=+9.116821293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:30.002296 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:30.002270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:30.002465 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:30.002360    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:30.002692 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:30.002678    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:30.002763 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:30.002749    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:30.002799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:30.002792    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:30.002865 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:30.002851    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:32.001848 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:32.001767    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:32.002276 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:32.002029    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:32.002276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:32.002174    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:32.002357 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:32.002292    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:32.002357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:32.002183    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:32.002447 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:32.002429    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:33.531917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:33.531877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:33.532397 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.532117    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:33.532397 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.532149    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:33.532397 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.532163    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:33.532397 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.532227    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:41.532205812 +0000 UTC m=+17.049885965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:33.632310 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:33.632273    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:33.632555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:33.632356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:33.632555 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.632441    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:33.632555 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.632503    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:33.632555 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.632529    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:41.632509822 +0000 UTC m=+17.150189961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:33.632555 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:33.632556    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:41.632538449 +0000 UTC m=+17.150218588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:34.002053    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:34.002173    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:34.002209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:34.002264    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:34.002310    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:34.002699 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:34.002366    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:36.001822 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:36.001580    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:36.002232 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:36.001594    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:36.002232 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:36.001604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:36.002232 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:36.001917    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:36.002232 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:36.001981    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:36.002232 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:36.002112    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:38.002096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:38.001998    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:38.002558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:38.002117    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:38.002558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:38.002149    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:38.002558 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:38.002130    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:38.002558 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:38.002231    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:38.002558 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:38.002312    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:40.001382 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:40.001344    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:40.001875 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:40.001439    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:40.001875 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:40.001467    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:40.001875 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:40.001472    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:40.001875 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:40.001531    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:40.001875 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:40.001587    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:41.596617 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:41.596585    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:41.597110 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.596724    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:41.597110 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.596743    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:41.597110 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.596754    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:41.597110 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.596806    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:57.596791525 +0000 UTC m=+33.114471665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:41.697917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:41.697883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:41.698107 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:41.697954    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:41.698107 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.698038    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:41.698107 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.698083    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:41.698107 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.698101    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:36:57.69808232 +0000 UTC m=+33.215762461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:41.698294 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:41.698131    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:36:57.698115085 +0000 UTC m=+33.215795229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:42.001866 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:42.001838    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:42.002109 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:42.001946    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:42.002109 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:42.001948    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:42.002109 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:42.002051    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:42.002282 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:42.002124    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:42.002282 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:42.002226    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:44.001421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:44.001391    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:44.001421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:44.001419    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:44.001876 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:44.001391    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:44.001876 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:44.001542    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:44.001876 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:44.001617    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:44.001876 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:44.001667    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:46.001784 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.001535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:46.002329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.001584    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:46.002329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.001589    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:46.002329 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:46.001959    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:46.002329 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:46.001854    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:46.002329 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:46.002023    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:46.220008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.219972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt" event={"ID":"eb597048-2bb8-4d1d-9e89-40b63492abc4","Type":"ContainerStarted","Data":"024fc940020e131e0b460c56006415ff130f6203e3ed2c008550ec71dddf075a"}
Sep 10 17:36:46.221490 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.221442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4szqx" event={"ID":"e5baaa61-1d31-4de2-b8bb-42cffda54965","Type":"ContainerStarted","Data":"af89da7a99228b251d717b69c3c0a35cb249469d747ae9b8139823140d7a2f78"}
Sep 10 17:36:46.224548 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224523    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"91ca4ee2c18f3b93a65b9cd315e823f915a30b4ba3ebed6771029c069880fd67"}
Sep 10 17:36:46.224677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224587    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"a892a09478233d4fd05d296d62d0da4004933ee7e83ce7699899156c0e3adf9f"}
Sep 10 17:36:46.224677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224627    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"d6d9bd232042b4aed9f67b8fe68e984d214bd4d123f4787418ae057d7e93fe45"}
Sep 10 17:36:46.224677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"8018f165f2d19e1d29bc11c8345d4d1af08793dcda2950d420146b9c4dc489dd"}
Sep 10 17:36:46.224812 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"e5be896cd4798fdad5b732dd5502ad93e6fac9918b14c9308a799f29a9ac557d"}
Sep 10 17:36:46.224812 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.224734    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"a3a2a32e83cf5ad0c6220971fe3cc0a70cc9d16d575cc3c634c4156f6c7fc509"}
Sep 10 17:36:46.226053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.226031    2571 generic.go:356] "Generic (PLEG): container finished" podID="de8ddf2b88624abfe9e09be17393198c" containerID="5ce3a0d9a2a5ba2e0332f791ed34ca74080163e980138abd17769ea10a57bd0a" exitCode=0
Sep 10 17:36:46.226140 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.226105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"de8ddf2b88624abfe9e09be17393198c","Type":"ContainerDied","Data":"5ce3a0d9a2a5ba2e0332f791ed34ca74080163e980138abd17769ea10a57bd0a"}
Sep 10 17:36:46.227539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.227521    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pttp4" event={"ID":"26515fca-4074-4d40-b42c-235ffebf01c3","Type":"ContainerStarted","Data":"a71f2def25ce253e3720a75ea2c97486fc6037eb6257225434a50832569cb70b"}
Sep 10 17:36:46.228903 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.228877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wpmlg" event={"ID":"cd4bebfe-d416-4305-abe5-3a401fa91ac8","Type":"ContainerStarted","Data":"cd6fe1db306296ee807eefa58755b8b564019a8cc7d73fd88b5e3896c660a9cb"}
Sep 10 17:36:46.230287 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.230268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" event={"ID":"6fbc923c8aeb003a0cb6b765d09dd22b","Type":"ContainerStarted","Data":"7ede344ae3bb978de764ab9333b29fd654e7bd67d7d6b910acaf5351fcdd4c2d"}
Sep 10 17:36:46.231687 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.231664    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtggj" event={"ID":"6adb3bcb-c09a-4291-8509-84188419571e","Type":"ContainerStarted","Data":"ef91463132cbffa691041ffe295c25369b68319336f7c90a94f57a5d1d1dd45f"}
Sep 10 17:36:46.233160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.233141    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="9464758d923b37d0276ae122213d1dcdd6b1d707535ed657b38b62e087849833" exitCode=0
Sep 10 17:36:46.233258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.233213    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"9464758d923b37d0276ae122213d1dcdd6b1d707535ed657b38b62e087849833"}
Sep 10 17:36:46.234669 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.234647    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-crqvl" event={"ID":"a7e5026e-a37f-45a4-a898-89f2c1052576","Type":"ContainerStarted","Data":"62632a68698ee2a170c75154dee9fc74fe87878932a628347098366e8f8fc562"}
Sep 10 17:36:46.236746 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.236703    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4szqx" podStartSLOduration=2.16150365 podStartE2EDuration="21.236691869s" totalImagesPullingTime="19.075188219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.236612146 +0000 UTC m=+21.754292304" watchObservedRunningTime="2026-09-10 17:36:46.236691869 +0000 UTC m=+21.754372039"
Sep 10 17:36:46.252558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.252526    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wpmlg" podStartSLOduration=2.200369247 podStartE2EDuration="21.252514995s" totalImagesPullingTime="19.052145748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.252307925 +0000 UTC m=+21.769988087" watchObservedRunningTime="2026-09-10 17:36:46.252514995 +0000 UTC m=+21.770195157"
Sep 10 17:36:46.288156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.288125    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rtggj" podStartSLOduration=2.247156913 podStartE2EDuration="21.28811441s" totalImagesPullingTime="19.040957497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.287636663 +0000 UTC m=+21.805316828" watchObservedRunningTime="2026-09-10 17:36:46.28811441 +0000 UTC m=+21.805794572"
Sep 10 17:36:46.301760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.301720    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-165.ec2.internal" podStartSLOduration=21.301703524 podStartE2EDuration="21.301703524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.30087551 +0000 UTC m=+21.818555673" watchObservedRunningTime="2026-09-10 17:36:46.301703524 +0000 UTC m=+21.819383689"
Sep 10 17:36:46.315446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.315409    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pttp4" podStartSLOduration=2.338397352 podStartE2EDuration="21.315394386s" totalImagesPullingTime="18.976997034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.314932325 +0000 UTC m=+21.832612500" watchObservedRunningTime="2026-09-10 17:36:46.315394386 +0000 UTC m=+21.833074549"
Sep 10 17:36:46.329124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.329060    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-crqvl" podStartSLOduration=2.55133979 podStartE2EDuration="21.329049358s" totalImagesPullingTime="18.777709568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:46.32872037 +0000 UTC m=+21.846400532" watchObservedRunningTime="2026-09-10 17:36:46.329049358 +0000 UTC m=+21.846729520"
Sep 10 17:36:46.419243 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.419212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:46.490293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.490268    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:46.543176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:46.543158    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crqvl_a7e5026e-a37f-45a4-a898-89f2c1052576/node-ca/0.log"
Sep 10 17:36:47.091501 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.091422    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 17:36:47.190638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.190608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.239033 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.239004    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5v9gh" event={"ID":"a79445c3-db0e-4891-832e-833f2c087675","Type":"ContainerStarted","Data":"51933646f51511d187b8407195c392106df4547a02a96e41ca560b01d808352c"}
Sep 10 17:36:47.241074 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.241048    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt" event={"ID":"eb597048-2bb8-4d1d-9e89-40b63492abc4","Type":"ContainerStarted","Data":"d27c65284e468b38b32d389e09e21d90aed15b74694d96fe4062ff51d41188f6"}
Sep 10 17:36:47.254096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.254060    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5v9gh" podStartSLOduration=3.134674285 podStartE2EDuration="22.254047777s" totalImagesPullingTime="19.119373492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:47.253618057 +0000 UTC m=+22.771298219" watchObservedRunningTime="2026-09-10 17:36:47.254047777 +0000 UTC m=+22.771727978"
Sep 10 17:36:47.458235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.458202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.558807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.558752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.713424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.713395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.812079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.812037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.918005 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.917963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:47.926020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.925920    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T17:36:47.091445748Z","UUID":"4b3e88ac-0ede-498c-9824-0f15c2993685","Handler":null,"Name":"","Endpoint":""}
Sep 10 17:36:47.927619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.927597    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 17:36:47.927619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:47.927633    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 17:36:48.002040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.002018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:48.002137 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.002018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:48.002265 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:48.002243    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:48.002321 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:48.002116    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:48.002321 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.002018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:48.002392 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:48.002336    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:48.151778 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.151359    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:48.246772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.246717    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt" event={"ID":"eb597048-2bb8-4d1d-9e89-40b63492abc4","Type":"ContainerStarted","Data":"5b71245d23ed16ca43aa6469e458613087dd9dd9bd19a1f2980ed85613862706"}
Sep 10 17:36:48.254009 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.253985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" event={"ID":"de8ddf2b88624abfe9e09be17393198c","Type":"ContainerStarted","Data":"cadeed43bc12e49e096c9b8a9a34b44c25429e98ccfb1484f713ee8c1f8b4ad9"}
Sep 10 17:36:48.281551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:48.281510    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htptt" podStartSLOduration=2.304249685 podStartE2EDuration="23.281495009s" totalImagesPullingTime="20.977245324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:48.266931458 +0000 UTC m=+23.784611621" watchObservedRunningTime="2026-09-10 17:36:48.281495009 +0000 UTC m=+23.799175171"
Sep 10 17:36:49.153830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:49.153559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:49.261254 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:49.261152    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"e82bc8371b51390555cecc840635a7129224b09231c1bcab85ffdaefb55e5b95"}
Sep 10 17:36:49.554856 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:49.554751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:49.763254 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:49.763209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:49.990276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:49.990238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:50.001240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:50.001211    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:50.001385 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:50.001259    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:50.001385 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:50.001335    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:50.001385 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:50.001353    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:50.001575 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:50.001410    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:50.001575 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:50.001496    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:50.159392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:50.159363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:50.395564 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:50.395532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:51.081903 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.081879    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:51.082336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.082315    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:51.097657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.097616    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-165.ec2.internal" podStartSLOduration=26.097604047 podStartE2EDuration="26.097604047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:48.28135422 +0000 UTC m=+23.799034383" watchObservedRunningTime="2026-09-10 17:36:51.097604047 +0000 UTC m=+26.615284208"
Sep 10 17:36:51.156167 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.155842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:51.268940 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.268911    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" event={"ID":"54b70f12-c630-4c43-863e-7dcfd6f0c68d","Type":"ContainerStarted","Data":"7bbaa175c5e1d6197dad70d4525c3d659bf3fff37a2c60d61cbdfb4cc26b6b93"}
Sep 10 17:36:51.269302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.269285    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:51.269376 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.269343    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:51.269638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.269616    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4szqx"
Sep 10 17:36:51.294987 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.294914    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w" podStartSLOduration=7.056226433 podStartE2EDuration="26.294899455s" totalImagesPullingTime="19.238673022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:36:51.294470743 +0000 UTC m=+26.812150899" watchObservedRunningTime="2026-09-10 17:36:51.294899455 +0000 UTC m=+26.812579617"
Sep 10 17:36:51.512465 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.512281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:51.610379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.610353    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:51.750579 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:51.750553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.001569 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.001539    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:52.001713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.001574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:52.001713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.001579    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:52.001713 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:52.001661    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:52.001871 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:52.001741    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:52.001871 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:52.001822    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:52.005258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.005232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.273217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.273151    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="9547fd692e5d9a7e3e0f1ecb3ee64d2fc0c05378f57fc090b209d1f96f121748" exitCode=0
Sep 10 17:36:52.273217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.273185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"9547fd692e5d9a7e3e0f1ecb3ee64d2fc0c05378f57fc090b209d1f96f121748"}
Sep 10 17:36:52.273802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.273781    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:52.273909 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.273858    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:52.291072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.291048    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:52.291380 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.291364    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:36:52.351035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.351011    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.409836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.409817    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.778360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.778331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.880717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.880692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:52.981301 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:52.981276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:53.046436 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:53.046370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:53.349472 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:53.349398    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:53.747387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:53.747361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:53.945095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:53.945061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:54.001240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.001180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:54.001240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.001180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:54.001240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.001201    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:54.001389 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:54.001298    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:54.001389 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:54.001354    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:54.001474 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:54.001429    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:54.279604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.279537    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="b74be4057f053f31deb0c8551625df4460a4bd847c085021bc0e100f96335b99" exitCode=0
Sep 10 17:36:54.279724 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.279633    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"b74be4057f053f31deb0c8551625df4460a4bd847c085021bc0e100f96335b99"}
Sep 10 17:36:54.547392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.547325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:54.703147 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:54.703121    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:55.152346 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.152284    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:55.284471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.284429    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="94b51e2a21e9f0c9a0fef3d969591c8e104e128c02e6ff85a274bf1648f0b95e" exitCode=0
Sep 10 17:36:55.284591 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.284484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"94b51e2a21e9f0c9a0fef3d969591c8e104e128c02e6ff85a274bf1648f0b95e"}
Sep 10 17:36:55.345759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.345736    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:55.750079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.750053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:55.947379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:55.947346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:56.001306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:56.001242    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:56.001436 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:56.001350    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:56.001436 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:56.001356    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:56.001436 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:56.001380    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:56.001622 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:56.001490    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:56.001622 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:56.001593    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:56.150374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:56.150326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:57.148389 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.148361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:57.546416 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.546331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:57.619918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.619890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:57.620100 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.620012    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 17:36:57.620100 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.620029    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 17:36:57.620100 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.620042    2571 projected.go:196] Error preparing data for projected volume kube-api-access-sl2sn for pod openshift-network-diagnostics/network-check-target-pj2gf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:57.620100 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.620094    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn podName:eec0661f-f759-4d48-9dcf-1d1934d210d3 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:29.620079623 +0000 UTC m=+65.137759763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sl2sn" (UniqueName: "kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn") pod "network-check-target-pj2gf" (UID: "eec0661f-f759-4d48-9dcf-1d1934d210d3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 17:36:57.720730 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.720700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:57.720913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.720763    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:57.720913 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.720865    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:57.720913 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.720876    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:57.721065 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.720933    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret podName:b00454b7-c84c-4280-87ca-39149012189f nodeName:}" failed. No retries permitted until 2026-09-10 17:37:29.72091439 +0000 UTC m=+65.238594531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret") pod "global-pull-secret-syncer-79kh2" (UID: "b00454b7-c84c-4280-87ca-39149012189f") : object "kube-system"/"original-pull-secret" not registered
Sep 10 17:36:57.721065 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:57.720953    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs podName:22631be7-f152-457e-a647-4dda83f32fa9 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:29.720941987 +0000 UTC m=+65.238622130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs") pod "network-metrics-daemon-czh2q" (UID: "22631be7-f152-457e-a647-4dda83f32fa9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 17:36:57.758059 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.758037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:57.953410 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:57.953378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:58.001293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:58.001268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:36:58.001447 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:58.001272    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:36:58.001447 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:58.001365    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:36:58.001599 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:58.001467    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:36:58.001599 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:58.001276    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:36:58.001695 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:36:58.001605    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:36:58.159741 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:58.159708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:36:59.071417 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:36:59.071379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:00.001338 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:00.001315    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:00.001730 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:00.001315    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:00.001730 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:00.001411    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:00.001730 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:00.001317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:00.001730 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:00.001502    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:00.001730 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:00.001581    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:00.299252 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:00.299189    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="244b706a6e0f0adc4b1e233e81a1727989ee63cb7a1f059bbacc290240b659de" exitCode=0
Sep 10 17:37:00.299374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:00.299263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"244b706a6e0f0adc4b1e233e81a1727989ee63cb7a1f059bbacc290240b659de"}
Sep 10 17:37:01.146216 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:01.146187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:01.304936 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:01.304910    2571 generic.go:356] "Generic (PLEG): container finished" podID="b135c9cb-01d1-4ac5-a31f-720cc697b74b" containerID="0b4e904e6bc383e888f97773137f301ca6961af1b4692d903b209628f8293e16" exitCode=0
Sep 10 17:37:01.305052 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:01.304946    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerDied","Data":"0b4e904e6bc383e888f97773137f301ca6961af1b4692d903b209628f8293e16"}
Sep 10 17:37:02.001633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:02.001606    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:02.001763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:02.001703    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:02.001805 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:02.001706    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:02.001845 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:02.001799    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:02.001845 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:02.001713    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:02.001905 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:02.001895    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:02.310666 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:02.310600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" event={"ID":"b135c9cb-01d1-4ac5-a31f-720cc697b74b","Type":"ContainerStarted","Data":"999ab998bfe2296cc10982a7d2df793c068f9f10aa680d21f914865c636a3142"}
Sep 10 17:37:02.333176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:02.333128    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t8l7c" podStartSLOduration=7.143683942 podStartE2EDuration="37.333115141s" totalImagesPullingTime="30.189431199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:02.331473254 +0000 UTC m=+37.849153410" watchObservedRunningTime="2026-09-10 17:37:02.333115141 +0000 UTC m=+37.850795334"
Sep 10 17:37:04.001731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:04.001695    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:04.002229 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:04.001708    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:04.002229 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:04.001793    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:04.002229 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:04.001917    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:04.002229 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:04.001707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:04.002229 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:04.002049    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:06.001996 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:06.001962    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:06.002387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:06.002059    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:06.002387 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:06.002062    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:06.002387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:06.002084    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:06.002387 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:06.002145    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:06.002387 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:06.002212    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:07.778984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:07.778939    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:07.836089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:07.836058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:08.001765 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:08.001737    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:08.001861 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:08.001770    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:08.001861 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:08.001852    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:08.001945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:08.001903    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:08.002012 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:08.001992    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:08.002100 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:08.002080    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:08.955667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:08.955639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.053817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.053790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.110974 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.110949    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.405812 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.405780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.501791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.501771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.571516 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.571488    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.700130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.700078    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.875128 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.875103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:09.937398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:09.937374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.001418 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.001356    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:10.001775 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:10.001471    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:10.001775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.001475    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:10.001775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.001490    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:10.001775 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:10.001548    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:10.001775 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:10.001618    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:10.271808 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.271745    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.338623 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.338592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.771099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.771075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.889096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.889070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:10.994334 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:10.994313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.084913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.084854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.157737 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.157707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.332886 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.332601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.402567 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.402523    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.482698 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.482660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.623239 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.623214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:11.903881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:11.903852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:12.002159 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.002131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:12.002269 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.002168    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:12.002269 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.002228    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:12.002269 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.002253    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:12.002430 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.002405    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:12.002516 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.002477    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:12.035114 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.034185    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79kh2"]
Sep 10 17:37:12.040330 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.040306    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pj2gf"]
Sep 10 17:37:12.041149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.041125    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-czh2q"]
Sep 10 17:37:12.291979 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.291946    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:12.332683 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.332665    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:12.332794 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.332688    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:12.332794 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.332766    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:12.332865 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.332793    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:12.332913 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.332891    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:12.333008 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:12.332989    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:12.357197 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:12.357166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:14.001836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:14.001805    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:14.001836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:14.001833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:14.002248 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:14.001903    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:14.002248 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:14.001965    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:14.002248 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:14.002062    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:14.002248 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:14.002094    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:14.043030 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:14.042999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:14.102074 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:14.102052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:15.337540 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:15.337503    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:15.390310 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:15.390282    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:16.001366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.001340    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:16.001366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.001359    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:16.001517 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:16.001465    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pj2gf" podUID="eec0661f-f759-4d48-9dcf-1d1934d210d3"
Sep 10 17:37:16.001517 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.001504    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:16.001629 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:16.001608    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-79kh2" podUID="b00454b7-c84c-4280-87ca-39149012189f"
Sep 10 17:37:16.001730 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:16.001710    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-czh2q" podUID="22631be7-f152-457e-a647-4dda83f32fa9"
Sep 10 17:37:16.272475 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.272258    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-165.ec2.internal" event="NodeReady"
Sep 10 17:37:16.272627 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.272509    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 17:37:16.313011 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.312987    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jzwqz"]
Sep 10 17:37:16.314774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.314758    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.316416 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.316393    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n7hsh"]
Sep 10 17:37:16.317208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.317047    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5h2r4\""
Sep 10 17:37:16.317208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.317074    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 17:37:16.317208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.317189    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 17:37:16.317378 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.317367    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 17:37:16.318358 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.318339    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.320836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.320819    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 17:37:16.320948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.320822    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 17:37:16.320948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.320895    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 17:37:16.321058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.320943    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gx7jr\""
Sep 10 17:37:16.321157 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.321137    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 17:37:16.334923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.334899    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jzwqz"]
Sep 10 17:37:16.335478 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.335445    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n7hsh"]
Sep 10 17:37:16.342149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.342128    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fwlg9"]
Sep 10 17:37:16.344069 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.344053    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.346684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.346667    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 17:37:16.346779 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.346688    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 17:37:16.346833 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.346804    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wtnr4\""
Sep 10 17:37:16.354155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.354135    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fwlg9"]
Sep 10 17:37:16.362883 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.362863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49be2ede-2ac7-470e-86ea-613d99ff8052-metrics-tls\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.362977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.362899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49be2ede-2ac7-470e-86ea-613d99ff8052-tmp-dir\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.362977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.362916    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpfc7\" (UniqueName: \"kubernetes.io/projected/49be2ede-2ac7-470e-86ea-613d99ff8052-kube-api-access-bpfc7\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.363068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.362989    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49be2ede-2ac7-470e-86ea-613d99ff8052-config-volume\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.363068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b858s\" (UniqueName: \"kubernetes.io/projected/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-api-access-b858s\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.363068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8f75fc7-1270-43c4-9efc-b9fc68497238-crio-socket\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.363068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363049    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.363068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8f75fc7-1270-43c4-9efc-b9fc68497238-data-volume\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.363237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577be349-f6d9-41a3-b751-b53e6ccf096e-cert\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.363237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363173    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zpz\" (UniqueName: \"kubernetes.io/projected/577be349-f6d9-41a3-b751-b53e6ccf096e-kube-api-access-m8zpz\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.363237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.363196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8f75fc7-1270-43c4-9efc-b9fc68497238-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.451922 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.451887    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:16.464040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464020    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8f75fc7-1270-43c4-9efc-b9fc68497238-crio-socket\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8f75fc7-1270-43c4-9efc-b9fc68497238-data-volume\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464096    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577be349-f6d9-41a3-b751-b53e6ccf096e-cert\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.464244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zpz\" (UniqueName: \"kubernetes.io/projected/577be349-f6d9-41a3-b751-b53e6ccf096e-kube-api-access-m8zpz\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.464244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8f75fc7-1270-43c4-9efc-b9fc68497238-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49be2ede-2ac7-470e-86ea-613d99ff8052-metrics-tls\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.464244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49be2ede-2ac7-470e-86ea-613d99ff8052-tmp-dir\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.464442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464259    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8f75fc7-1270-43c4-9efc-b9fc68497238-crio-socket\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464738    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49be2ede-2ac7-470e-86ea-613d99ff8052-tmp-dir\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.464837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464746    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8f75fc7-1270-43c4-9efc-b9fc68497238-data-volume\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.464837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpfc7\" (UniqueName: \"kubernetes.io/projected/49be2ede-2ac7-470e-86ea-613d99ff8052-kube-api-access-bpfc7\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.464920 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49be2ede-2ac7-470e-86ea-613d99ff8052-config-volume\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.464920 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b858s\" (UniqueName: \"kubernetes.io/projected/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-api-access-b858s\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.465093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.464999    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.465433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.465412    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/49be2ede-2ac7-470e-86ea-613d99ff8052-config-volume\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.468298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.468281    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8f75fc7-1270-43c4-9efc-b9fc68497238-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.468355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.468311    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/49be2ede-2ac7-470e-86ea-613d99ff8052-metrics-tls\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.468355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.468317    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577be349-f6d9-41a3-b751-b53e6ccf096e-cert\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.472177 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.472157    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b858s\" (UniqueName: \"kubernetes.io/projected/f8f75fc7-1270-43c4-9efc-b9fc68497238-kube-api-access-b858s\") pod \"insights-runtime-extractor-n7hsh\" (UID: \"f8f75fc7-1270-43c4-9efc-b9fc68497238\") " pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.472387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.472369    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpfc7\" (UniqueName: \"kubernetes.io/projected/49be2ede-2ac7-470e-86ea-613d99ff8052-kube-api-access-bpfc7\") pod \"dns-default-fwlg9\" (UID: \"49be2ede-2ac7-470e-86ea-613d99ff8052\") " pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.472650 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.472631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zpz\" (UniqueName: \"kubernetes.io/projected/577be349-f6d9-41a3-b751-b53e6ccf096e-kube-api-access-m8zpz\") pod \"ingress-canary-jzwqz\" (UID: \"577be349-f6d9-41a3-b751-b53e6ccf096e\") " pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.554908 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.554853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:16.627437 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.627417    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jzwqz"
Sep 10 17:37:16.630923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.630910    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n7hsh"
Sep 10 17:37:16.652291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.652250    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:16.919051 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.919022    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jzwqz"]
Sep 10 17:37:16.921221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.921195    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n7hsh"]
Sep 10 17:37:16.922935 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:16.922914    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577be349_f6d9_41a3_b751_b53e6ccf096e.slice/crio-c165c3cc669df1f36e856dc66d8b02f2268814add6a07704d8266d3c1c2c0360 WatchSource:0}: Error finding container c165c3cc669df1f36e856dc66d8b02f2268814add6a07704d8266d3c1c2c0360: Status 404 returned error can't find the container with id c165c3cc669df1f36e856dc66d8b02f2268814add6a07704d8266d3c1c2c0360
Sep 10 17:37:16.925176 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:16.925156    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f75fc7_1270_43c4_9efc_b9fc68497238.slice/crio-dcd24a6e690a10ee21d236d0a98097e5772e7e4a16235517ef6820a95c1fd222 WatchSource:0}: Error finding container dcd24a6e690a10ee21d236d0a98097e5772e7e4a16235517ef6820a95c1fd222: Status 404 returned error can't find the container with id dcd24a6e690a10ee21d236d0a98097e5772e7e4a16235517ef6820a95c1fd222
Sep 10 17:37:16.939541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:16.939519    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fwlg9"]
Sep 10 17:37:16.943217 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:16.943186    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49be2ede_2ac7_470e_86ea_613d99ff8052.slice/crio-a166300ec5c9b53d1177a35ea85aa1b6d10496ff42bbb9f66addcd2d680bd697 WatchSource:0}: Error finding container a166300ec5c9b53d1177a35ea85aa1b6d10496ff42bbb9f66addcd2d680bd697: Status 404 returned error can't find the container with id a166300ec5c9b53d1177a35ea85aa1b6d10496ff42bbb9f66addcd2d680bd697
Sep 10 17:37:17.344754 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:17.344670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwlg9" event={"ID":"49be2ede-2ac7-470e-86ea-613d99ff8052","Type":"ContainerStarted","Data":"a166300ec5c9b53d1177a35ea85aa1b6d10496ff42bbb9f66addcd2d680bd697"}
Sep 10 17:37:17.346490 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:17.346292    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7hsh" event={"ID":"f8f75fc7-1270-43c4-9efc-b9fc68497238","Type":"ContainerStarted","Data":"0cb3b1930895569af7b4fb1aaea3660a0cb6aca7f8b8248c282020c65775e0af"}
Sep 10 17:37:17.346603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:17.346517    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7hsh" event={"ID":"f8f75fc7-1270-43c4-9efc-b9fc68497238","Type":"ContainerStarted","Data":"dcd24a6e690a10ee21d236d0a98097e5772e7e4a16235517ef6820a95c1fd222"}
Sep 10 17:37:17.347470 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:17.347434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jzwqz" event={"ID":"577be349-f6d9-41a3-b751-b53e6ccf096e","Type":"ContainerStarted","Data":"c165c3cc669df1f36e856dc66d8b02f2268814add6a07704d8266d3c1c2c0360"}
Sep 10 17:37:18.001786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.001713    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:18.001786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.001734    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:18.001786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.001735    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:18.004761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.004736    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:37:18.005640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.005616    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:37:18.005640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.005631    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:37:18.005963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.005917    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:37:18.006031 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.005989    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lms2\""
Sep 10 17:37:18.006031 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.006012    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh4f6\""
Sep 10 17:37:18.275219 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.275029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:18.354149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.354113    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7hsh" event={"ID":"f8f75fc7-1270-43c4-9efc-b9fc68497238","Type":"ContainerStarted","Data":"ea23b64172304fa7703b580b904f659d4fe030a2e854ea106f88d2a8ed430a12"}
Sep 10 17:37:18.375911 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.375872    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:18.898702 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:18.898668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:19.000038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.000000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:19.962380 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.962357    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:37:19.964708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.964668    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.969441 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969213    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 17:37:19.969441 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969249    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 17:37:19.969441 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969323    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 17:37:19.969441 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969334    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 17:37:19.969759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969574    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 17:37:19.969759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969664    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bc6nq\""
Sep 10 17:37:19.969859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969806    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 17:37:19.969906 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.969861    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 17:37:19.977191 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.977172    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:37:19.992643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.992783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992654    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.992783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992678    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xb4\" (UniqueName: \"kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.992783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992707    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.992783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992731    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:19.992783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:19.992759    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.093913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.093913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093878    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093951    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093965    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xb4\" (UniqueName: \"kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.093981    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.094711    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.094755    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.094859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.094768    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.097256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.097234    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.097347 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.097245    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.102788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.102765    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xb4\" (UniqueName: \"kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4\") pod \"console-7cf967bc96-qzf7p\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") " pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.165488 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.165443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:20.230518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.230489    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:20.278223 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.278206    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:20.362189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.362155    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwlg9" event={"ID":"49be2ede-2ac7-470e-86ea-613d99ff8052","Type":"ContainerStarted","Data":"427abb6ace88ccb419f5b9202e77417ddf2736ddbb61a3d6a938d3c7b691076c"}
Sep 10 17:37:20.362313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.362224    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:20.362313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.362256    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwlg9" event={"ID":"49be2ede-2ac7-470e-86ea-613d99ff8052","Type":"ContainerStarted","Data":"373f886c148bc805da329297c3fd0ab9cb2bbcd3ee1b700d2b6a42c6f56f5fd7"}
Sep 10 17:37:20.364224 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.364202    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7hsh" event={"ID":"f8f75fc7-1270-43c4-9efc-b9fc68497238","Type":"ContainerStarted","Data":"1d7f7fdc6441915f3f619572665ec9988278bc3e39b8b35bdc6d5fbe56438db1"}
Sep 10 17:37:20.365649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.365628    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jzwqz" event={"ID":"577be349-f6d9-41a3-b751-b53e6ccf096e","Type":"ContainerStarted","Data":"58bb6093cd3a5a117fa37b1931d4cb2b87c79fd431daadd068b75af308007cc3"}
Sep 10 17:37:20.380060 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.380010    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fwlg9" podStartSLOduration=1.527939065 podStartE2EDuration="4.379992793s" totalImagesPullingTime="2.852053728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:20.379737336 +0000 UTC m=+55.897417500" watchObservedRunningTime="2026-09-10 17:37:20.379992793 +0000 UTC m=+55.897673001"
Sep 10 17:37:20.395865 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.395825    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jzwqz" podStartSLOduration=1.5174834910000001 podStartE2EDuration="4.395813699s" totalImagesPullingTime="2.878330208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:20.39522957 +0000 UTC m=+55.912909733" watchObservedRunningTime="2026-09-10 17:37:20.395813699 +0000 UTC m=+55.913493861"
Sep 10 17:37:20.413868 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.413831    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n7hsh" podStartSLOduration=1.681052376 podStartE2EDuration="4.413818741s" totalImagesPullingTime="2.732766365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:37:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:20.413624003 +0000 UTC m=+55.931304165" watchObservedRunningTime="2026-09-10 17:37:20.413818741 +0000 UTC m=+55.931498904"
Sep 10 17:37:20.547791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:20.547767    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:37:20.550713 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:20.550687    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5097fae6_74c8_4aea_bbe7_9d6e6bfa4f7f.slice/crio-6999eb2b0f960dd07a13926e3cbbee2e0004397a52b71eaf8a399fb336445137 WatchSource:0}: Error finding container 6999eb2b0f960dd07a13926e3cbbee2e0004397a52b71eaf8a399fb336445137: Status 404 returned error can't find the container with id 6999eb2b0f960dd07a13926e3cbbee2e0004397a52b71eaf8a399fb336445137
Sep 10 17:37:21.134250 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:21.134219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:21.235828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:21.235801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:21.335007 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:21.334984    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:21.370009 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:21.369983    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf967bc96-qzf7p" event={"ID":"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f","Type":"ContainerStarted","Data":"6999eb2b0f960dd07a13926e3cbbee2e0004397a52b71eaf8a399fb336445137"}
Sep 10 17:37:21.399996 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:21.399942    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:23.429099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:23.429055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:24.291044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:24.290974    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvl9w"
Sep 10 17:37:24.385412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:24.385381    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf967bc96-qzf7p" event={"ID":"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f","Type":"ContainerStarted","Data":"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"}
Sep 10 17:37:24.404508 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:24.404436    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cf967bc96-qzf7p" podStartSLOduration=2.005389474 podStartE2EDuration="5.404421523s" totalImagesPullingTime="3.399032049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:24.40330296 +0000 UTC m=+59.920983123" watchObservedRunningTime="2026-09-10 17:37:24.404421523 +0000 UTC m=+59.922101685"
Sep 10 17:37:24.407626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:24.407606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:24.508719 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:24.508693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:25.173801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:25.173770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:25.233082 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:25.233053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.277214 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.277179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.519742 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.519704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.527331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.527261    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"]
Sep 10 17:37:26.530202 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.530184    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.532947 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.532926    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 17:37:26.533089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.532966    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 17:37:26.533089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.532980    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 17:37:26.533089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.532991    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 17:37:26.533089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.533050    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 17:37:26.533298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.533190    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cxffk\""
Sep 10 17:37:26.543843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.543820    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"]
Sep 10 17:37:26.550614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.550591    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d9l5j"]
Sep 10 17:37:26.552832 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.552816    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.555725 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.555703    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 17:37:26.555942 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.555927    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 17:37:26.556200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.556186    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 17:37:26.556328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.556311    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rtt87\""
Sep 10 17:37:26.608129 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.608104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.639236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.639341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639241    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-wtmp\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e5825a7-1574-44b5-8476-093ab3ca6d4c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.639341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639277    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pcw\" (UniqueName: \"kubernetes.io/projected/3e5825a7-1574-44b5-8476-093ab3ca6d4c-kube-api-access-k4pcw\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.639541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639371    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-metrics-client-ca\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639404    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-textfile\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639587    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-sys\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-root\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.639685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639669    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.639868 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.639696    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56z76\" (UniqueName: \"kubernetes.io/projected/04051485-b57b-4e87-8e76-4ba3cd3d5721-kube-api-access-56z76\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.672693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.672673    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.740294 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740269    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-metrics-client-ca\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740325    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-textfile\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740367    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-sys\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740387    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-root\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740420    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740448    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740494    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-sys\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740506    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56z76\" (UniqueName: \"kubernetes.io/projected/04051485-b57b-4e87-8e76-4ba3cd3d5721-kube-api-access-56z76\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740510    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-root\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740601    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740636    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-wtmp\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e5825a7-1574-44b5-8476-093ab3ca6d4c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pcw\" (UniqueName: \"kubernetes.io/projected/3e5825a7-1574-44b5-8476-093ab3ca6d4c-kube-api-access-k4pcw\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740878    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-wtmp\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.740916 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:26.740894    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:26.740960    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls podName:04051485-b57b-4e87-8e76-4ba3cd3d5721 nodeName:}" failed. No retries permitted until 2026-09-10 17:37:27.240939189 +0000 UTC m=+62.758619377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls") pod "node-exporter-d9l5j" (UID: "04051485-b57b-4e87-8e76-4ba3cd3d5721") : secret "node-exporter-tls" not found
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.740967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-textfile\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:26.740975    2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:37:26.741051    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls podName:3e5825a7-1574-44b5-8476-093ab3ca6d4c nodeName:}" failed. No retries permitted until 2026-09-10 17:37:27.241031763 +0000 UTC m=+62.758711903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-cf7rr" (UID: "3e5825a7-1574-44b5-8476-093ab3ca6d4c") : secret "openshift-state-metrics-tls" not found
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.741099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-metrics-client-ca\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.741384 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.741347    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-accelerators-collector-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.741640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.741620    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e5825a7-1574-44b5-8476-093ab3ca6d4c-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.743337 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.743310    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.743568 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.743548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.751299 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.751272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56z76\" (UniqueName: \"kubernetes.io/projected/04051485-b57b-4e87-8e76-4ba3cd3d5721-kube-api-access-56z76\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:26.751588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.751570    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pcw\" (UniqueName: \"kubernetes.io/projected/3e5825a7-1574-44b5-8476-093ab3ca6d4c-kube-api-access-k4pcw\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:26.786142 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.786072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.886338 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.886314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:26.986040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:26.986014    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.228120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.228090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.243984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.243960    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:27.244076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.244049    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:27.246798 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.246759    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04051485-b57b-4e87-8e76-4ba3cd3d5721-node-exporter-tls\") pod \"node-exporter-d9l5j\" (UID: \"04051485-b57b-4e87-8e76-4ba3cd3d5721\") " pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:27.246983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.246958    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e5825a7-1574-44b5-8476-093ab3ca6d4c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-cf7rr\" (UID: \"3e5825a7-1574-44b5-8476-093ab3ca6d4c\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:27.329890 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.329863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.440332 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.440305    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"
Sep 10 17:37:27.462051 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.462029    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d9l5j"
Sep 10 17:37:27.471540 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:27.471516    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04051485_b57b_4e87_8e76_4ba3cd3d5721.slice/crio-0d2af80c2698771c5adbcad96b608861b19804a632a8b56921e18899e989ccaa WatchSource:0}: Error finding container 0d2af80c2698771c5adbcad96b608861b19804a632a8b56921e18899e989ccaa: Status 404 returned error can't find the container with id 0d2af80c2698771c5adbcad96b608861b19804a632a8b56921e18899e989ccaa
Sep 10 17:37:27.491871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.491792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.574415 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.574383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.581991 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.581965    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:37:27.584937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.584921    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.587331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587306    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 17:37:27.587467 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587344    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 17:37:27.587467 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587352    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 17:37:27.587467 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587446    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 17:37:27.587637 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587490    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 17:37:27.587637 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587603    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 17:37:27.587732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587687    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 17:37:27.587732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587687    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-crtdc\""
Sep 10 17:37:27.587850 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587837    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 17:37:27.587963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.587931    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 17:37:27.601811 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.601793    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:37:27.647666 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647860 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647860 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647781    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.647962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647936    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.648076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.648076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.647995    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.648076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.648013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.648076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.648049    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qm9x\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.648209 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.648075    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.697573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.697547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:27.737488 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.737467    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr"]
Sep 10 17:37:27.740431 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:27.740403    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e5825a7_1574_44b5_8476_093ab3ca6d4c.slice/crio-9d2eea66af93697dc575a9acae9400ee43e44df20c6bcf87422e75218ee4bce7 WatchSource:0}: Error finding container 9d2eea66af93697dc575a9acae9400ee43e44df20c6bcf87422e75218ee4bce7: Status 404 returned error can't find the container with id 9d2eea66af93697dc575a9acae9400ee43e44df20c6bcf87422e75218ee4bce7
Sep 10 17:37:27.749270 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749350 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749282    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749350 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749350 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qm9x\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749511    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749600    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749656    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.749731 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.749722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.750509 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.750487    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752682 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752682 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752682 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752544    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752875 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752688    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752932 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752883    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.752932 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.752511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.753643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.753268    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.753643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.753308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.755319 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.755292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.755400 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.755339    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.755744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.755724    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.758264 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.758242    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qm9x\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x\") pod \"alertmanager-main-0\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:27.894067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:27.894035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:37:28.097786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.097715    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:37:28.099946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.099922    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.107065 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.107045    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 17:37:28.110788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.110765    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:37:28.149705 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.149684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:28.152892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.152871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.152985 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.152907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.152985 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.152974    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4tw4\" (UniqueName: \"kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.153090 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.153057    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.153139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.153087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.153139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.153115    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.153217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.153147    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.169738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.169720    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:37:28.171920 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:28.171896    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e1877a_2551_4d05_95a8_69d014bcb3f4.slice/crio-0db38774ed37162466371722d02740c8b757e0e95119ab031ac43f10e55ada8a WatchSource:0}: Error finding container 0db38774ed37162466371722d02740c8b757e0e95119ab031ac43f10e55ada8a: Status 404 returned error can't find the container with id 0db38774ed37162466371722d02740c8b757e0e95119ab031ac43f10e55ada8a
Sep 10 17:37:28.213301 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.213269    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:28.253850 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.253825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.253951 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.253861    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.253951 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.253891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.253951 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.253917    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.253952    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254084    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254196 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254150    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4tw4\" (UniqueName: \"kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254627    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254789    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254817    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.254990 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.254830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.256705 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.256685    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.257135 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.257114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.261959 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.261938    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4tw4\" (UniqueName: \"kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4\") pod \"console-5985d64575-wbf8p\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") " pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.405313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.405067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr" event={"ID":"3e5825a7-1574-44b5-8476-093ab3ca6d4c","Type":"ContainerStarted","Data":"9d229e7353e1ba5ee8f6c3f2815b312773df147595b912acae008b19b3ddb44a"}
Sep 10 17:37:28.405313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.405158    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr" event={"ID":"3e5825a7-1574-44b5-8476-093ab3ca6d4c","Type":"ContainerStarted","Data":"d774af95704a95d549445c36cf38643e29b9b99fcdd7f86f98c30c35f7356a6d"}
Sep 10 17:37:28.405313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.405182    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr" event={"ID":"3e5825a7-1574-44b5-8476-093ab3ca6d4c","Type":"ContainerStarted","Data":"9d2eea66af93697dc575a9acae9400ee43e44df20c6bcf87422e75218ee4bce7"}
Sep 10 17:37:28.407380 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.407352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"0db38774ed37162466371722d02740c8b757e0e95119ab031ac43f10e55ada8a"}
Sep 10 17:37:28.408942 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.408908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9l5j" event={"ID":"04051485-b57b-4e87-8e76-4ba3cd3d5721","Type":"ContainerStarted","Data":"0d2af80c2698771c5adbcad96b608861b19804a632a8b56921e18899e989ccaa"}
Sep 10 17:37:28.411325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.411302    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:28.885642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:28.885570    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:37:29.121744 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:29.121714    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6db0b1_5287_45af_a8ac_2151402910c3.slice/crio-27f45b305a9c7e157fd87ba0b5b82cd3bb3c503835b18eed602605fca7f37604 WatchSource:0}: Error finding container 27f45b305a9c7e157fd87ba0b5b82cd3bb3c503835b18eed602605fca7f37604: Status 404 returned error can't find the container with id 27f45b305a9c7e157fd87ba0b5b82cd3bb3c503835b18eed602605fca7f37604
Sep 10 17:37:29.413877 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.413811    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5985d64575-wbf8p" event={"ID":"ac6db0b1-5287-45af-a8ac-2151402910c3","Type":"ContainerStarted","Data":"27f45b305a9c7e157fd87ba0b5b82cd3bb3c503835b18eed602605fca7f37604"}
Sep 10 17:37:29.444753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.444728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:29.509933 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.509895    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:29.596888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.596867    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:29.667379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.667329    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:29.670004 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.669978    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 17:37:29.680882 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.680861    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 17:37:29.690905 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.690884    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl2sn\" (UniqueName: \"kubernetes.io/projected/eec0661f-f759-4d48-9dcf-1d1934d210d3-kube-api-access-sl2sn\") pod \"network-check-target-pj2gf\" (UID: \"eec0661f-f759-4d48-9dcf-1d1934d210d3\") " pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:29.718855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.718763    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sh4f6\""
Sep 10 17:37:29.727575 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.727234    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:29.768494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.768358    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:29.768494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.768398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:29.771315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.771110    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 17:37:29.771315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.771236    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 17:37:29.782596 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.782553    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b00454b7-c84c-4280-87ca-39149012189f-original-pull-secret\") pod \"global-pull-secret-syncer-79kh2\" (UID: \"b00454b7-c84c-4280-87ca-39149012189f\") " pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:29.783299 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.783256    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/22631be7-f152-457e-a647-4dda83f32fa9-metrics-certs\") pod \"network-metrics-daemon-czh2q\" (UID: \"22631be7-f152-457e-a647-4dda83f32fa9\") " pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:29.917218 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:29.917196    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:30.020218 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.020191    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pj2gf"]
Sep 10 17:37:30.025037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.025017    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-79kh2"
Sep 10 17:37:30.033788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.033770    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7lms2\""
Sep 10 17:37:30.042047 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.042033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-czh2q"
Sep 10 17:37:30.052200 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:30.052174    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec0661f_f759_4d48_9dcf_1d1934d210d3.slice/crio-225547d9ec51b744b4d2222abc9d9a7d0d8f4599204396ff75dd1fec7e269b3e WatchSource:0}: Error finding container 225547d9ec51b744b4d2222abc9d9a7d0d8f4599204396ff75dd1fec7e269b3e: Status 404 returned error can't find the container with id 225547d9ec51b744b4d2222abc9d9a7d0d8f4599204396ff75dd1fec7e269b3e
Sep 10 17:37:30.262593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.262529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:30.279071 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.279048    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:30.279175 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.279101    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:30.283566 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.283547    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:30.308015 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.307994    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-79kh2"]
Sep 10 17:37:30.310938 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:30.310917    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00454b7_c84c_4280_87ca_39149012189f.slice/crio-3ed33b14537d7c8864f7960289a4a9c17463793987471b35ff629e6d60d57ef2 WatchSource:0}: Error finding container 3ed33b14537d7c8864f7960289a4a9c17463793987471b35ff629e6d60d57ef2: Status 404 returned error can't find the container with id 3ed33b14537d7c8864f7960289a4a9c17463793987471b35ff629e6d60d57ef2
Sep 10 17:37:30.343235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.343210    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-czh2q"]
Sep 10 17:37:30.346890 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:30.346867    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22631be7_f152_457e_a647_4dda83f32fa9.slice/crio-3cb60c9cb6ce80378ffaee32ece13a2a5f7021bc10d50fb9f50ff559c0e72271 WatchSource:0}: Error finding container 3cb60c9cb6ce80378ffaee32ece13a2a5f7021bc10d50fb9f50ff559c0e72271: Status 404 returned error can't find the container with id 3cb60c9cb6ce80378ffaee32ece13a2a5f7021bc10d50fb9f50ff559c0e72271
Sep 10 17:37:30.347105 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.347087    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:30.372149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.372128    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fwlg9"
Sep 10 17:37:30.419700 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.419611    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pj2gf" event={"ID":"eec0661f-f759-4d48-9dcf-1d1934d210d3","Type":"ContainerStarted","Data":"225547d9ec51b744b4d2222abc9d9a7d0d8f4599204396ff75dd1fec7e269b3e"}
Sep 10 17:37:30.422187 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.422157    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr" event={"ID":"3e5825a7-1574-44b5-8476-093ab3ca6d4c","Type":"ContainerStarted","Data":"8bf6ae3eb2b95b792639a55b82e3b9399de7d6e8c9f7e143a33113f055b28b04"}
Sep 10 17:37:30.424572 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.424543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5985d64575-wbf8p" event={"ID":"ac6db0b1-5287-45af-a8ac-2151402910c3","Type":"ContainerStarted","Data":"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"}
Sep 10 17:37:30.426649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.426622    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc" exitCode=0
Sep 10 17:37:30.426751 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.426670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc"}
Sep 10 17:37:30.430291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.430267    2571 generic.go:356] "Generic (PLEG): container finished" podID="04051485-b57b-4e87-8e76-4ba3cd3d5721" containerID="33d1f0c603be41707359b5a4bd241a721459ca9e5ecc7b7d92e97a1ab68cd67b" exitCode=0
Sep 10 17:37:30.430400 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.430350    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9l5j" event={"ID":"04051485-b57b-4e87-8e76-4ba3cd3d5721","Type":"ContainerDied","Data":"33d1f0c603be41707359b5a4bd241a721459ca9e5ecc7b7d92e97a1ab68cd67b"}
Sep 10 17:37:30.432210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.432186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czh2q" event={"ID":"22631be7-f152-457e-a647-4dda83f32fa9","Type":"ContainerStarted","Data":"3cb60c9cb6ce80378ffaee32ece13a2a5f7021bc10d50fb9f50ff559c0e72271"}
Sep 10 17:37:30.434543 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.434521    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79kh2" event={"ID":"b00454b7-c84c-4280-87ca-39149012189f","Type":"ContainerStarted","Data":"3ed33b14537d7c8864f7960289a4a9c17463793987471b35ff629e6d60d57ef2"}
Sep 10 17:37:30.439078 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.439048    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:37:30.466236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.465936    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-cf7rr" podStartSLOduration=2.992989189 podStartE2EDuration="4.465894988s" totalImagesPullingTime="1.472905799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:30.447061043 +0000 UTC m=+65.964741205" watchObservedRunningTime="2026-09-10 17:37:30.465894988 +0000 UTC m=+65.983575151"
Sep 10 17:37:30.526846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.526715    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5985d64575-wbf8p" podStartSLOduration=2.5266950809999997 podStartE2EDuration="2.526695081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:30.52604192 +0000 UTC m=+66.043722086" watchObservedRunningTime="2026-09-10 17:37:30.526695081 +0000 UTC m=+66.044375244"
Sep 10 17:37:30.706852 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.706814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:30.767148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.767071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:30.935174 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.935143    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c9d989b66-nmpgs"]
Sep 10 17:37:30.937718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.937695    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.941040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.940642    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 17:37:30.941040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.940687    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 17:37:30.941040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.940828    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 17:37:30.941040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.940892    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zbnlk\""
Sep 10 17:37:30.941603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.941556    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 17:37:30.941822 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.941801    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-129lkd3n482f6\""
Sep 10 17:37:30.950120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.948754    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c9d989b66-nmpgs"]
Sep 10 17:37:30.979332 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-client-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979349    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-metrics-server-audit-profiles\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979389    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-tls\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979440    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ce5a5c11-d93e-4ca5-8f16-0393e2246481-audit-log\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979504    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-client-certs\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76zl\" (UniqueName: \"kubernetes.io/projected/ce5a5c11-d93e-4ca5-8f16-0393e2246481-kube-api-access-p76zl\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:30.979728 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:30.979571    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080138 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080050    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ce5a5c11-d93e-4ca5-8f16-0393e2246481-audit-log\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080138 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080097    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-client-certs\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080138 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76zl\" (UniqueName: \"kubernetes.io/projected/ce5a5c11-d93e-4ca5-8f16-0393e2246481-kube-api-access-p76zl\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080206    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-client-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-metrics-server-audit-profiles\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080270    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-tls\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.080785 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.080576    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ce5a5c11-d93e-4ca5-8f16-0393e2246481-audit-log\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.081284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.081219    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.081732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.081705    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ce5a5c11-d93e-4ca5-8f16-0393e2246481-metrics-server-audit-profiles\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.084396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.084317    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-client-ca-bundle\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.085080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.085018    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-tls\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.086190 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.086141    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ce5a5c11-d93e-4ca5-8f16-0393e2246481-secret-metrics-server-client-certs\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.097918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.097877    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76zl\" (UniqueName: \"kubernetes.io/projected/ce5a5c11-d93e-4ca5-8f16-0393e2246481-kube-api-access-p76zl\") pod \"metrics-server-c9d989b66-nmpgs\" (UID: \"ce5a5c11-d93e-4ca5-8f16-0393e2246481\") " pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.251797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.251752    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:31.305769 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.304527    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"]
Sep 10 17:37:31.307341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.307199    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.308125 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.307613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:31.310896 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.309784    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l9w4t\""
Sep 10 17:37:31.310896 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.310018    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 17:37:31.321236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.321210    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"]
Sep 10 17:37:31.364366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.364336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:31.383918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.383892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cswr\" (UniqueName: \"kubernetes.io/projected/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-kube-api-access-6cswr\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.384023 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.383930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.443325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.443293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9l5j" event={"ID":"04051485-b57b-4e87-8e76-4ba3cd3d5721","Type":"ContainerStarted","Data":"26737f7fcf4e957610d9f09b442ed9b521b5aa6e8a0453a20b1698372867be97"}
Sep 10 17:37:31.443777 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.443353    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d9l5j" event={"ID":"04051485-b57b-4e87-8e76-4ba3cd3d5721","Type":"ContainerStarted","Data":"88b7a6e58be9b6bd116827bc4e752cfb0f368a5f2969c3df96be5c1c59a1b16a"}
Sep 10 17:37:31.463408 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.463365    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d9l5j" podStartSLOduration=3.5229949339999997 podStartE2EDuration="5.463350127s" totalImagesPullingTime="1.852548837s" totalInitContainerRuntime="87.806356ms" isStatefulPod=true podCreationTimestamp="2026-09-10 17:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:31.461663885 +0000 UTC m=+66.979344062" watchObservedRunningTime="2026-09-10 17:37:31.463350127 +0000 UTC m=+66.981030288"
Sep 10 17:37:31.484502 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.484468    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cswr\" (UniqueName: \"kubernetes.io/projected/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-kube-api-access-6cswr\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.484646 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.484558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.490972 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.490947    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.492681 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.492661    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cswr\" (UniqueName: \"kubernetes.io/projected/0fe4d4d9-cbea-4857-a3f0-5faacd4c0564-kube-api-access-6cswr\") pod \"monitoring-plugin-85bbbbbf4-jjjcs\" (UID: \"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.524439 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.524370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:31.622152 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.621875    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:31.745835 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.745806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c9d989b66-nmpgs"]
Sep 10 17:37:31.751604 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:31.751577    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5a5c11_d93e_4ca5_8f16_0393e2246481.slice/crio-162ff64922f6b08a3fe1b20e9ed3620500aaaf0fd1772be50348635adc85a58e WatchSource:0}: Error finding container 162ff64922f6b08a3fe1b20e9ed3620500aaaf0fd1772be50348635adc85a58e: Status 404 returned error can't find the container with id 162ff64922f6b08a3fe1b20e9ed3620500aaaf0fd1772be50348635adc85a58e
Sep 10 17:37:31.922205 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:31.922129    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"]
Sep 10 17:37:32.135967 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.135934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:32.196132 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.196024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:32.360938 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:32.360902    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe4d4d9_cbea_4857_a3f0_5faacd4c0564.slice/crio-14ef30a5bce2940a2176d91c344573ced60a0feb180a93c5519caa9a35aff641 WatchSource:0}: Error finding container 14ef30a5bce2940a2176d91c344573ced60a0feb180a93c5519caa9a35aff641: Status 404 returned error can't find the container with id 14ef30a5bce2940a2176d91c344573ced60a0feb180a93c5519caa9a35aff641
Sep 10 17:37:32.449473 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.449377    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czh2q" event={"ID":"22631be7-f152-457e-a647-4dda83f32fa9","Type":"ContainerStarted","Data":"02c743e4e7a9b6c6f2333e8db9991dffbfb72a584e5f9cc503296b92f9be9183"}
Sep 10 17:37:32.449852 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.449501    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-czh2q" event={"ID":"22631be7-f152-457e-a647-4dda83f32fa9","Type":"ContainerStarted","Data":"6083b76fce14856016b143cafbf409cff99fd8d9467cec363a7ce2aaa27b1193"}
Sep 10 17:37:32.450926 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.450892    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs" event={"ID":"ce5a5c11-d93e-4ca5-8f16-0393e2246481","Type":"ContainerStarted","Data":"162ff64922f6b08a3fe1b20e9ed3620500aaaf0fd1772be50348635adc85a58e"}
Sep 10 17:37:32.452158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.452135    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs" event={"ID":"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564","Type":"ContainerStarted","Data":"14ef30a5bce2940a2176d91c344573ced60a0feb180a93c5519caa9a35aff641"}
Sep 10 17:37:32.469856 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.469801    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-czh2q" podStartSLOduration=66.323929962 podStartE2EDuration="1m7.469781354s" totalImagesPullingTime="1.145851392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:32.468358439 +0000 UTC m=+67.986038601" watchObservedRunningTime="2026-09-10 17:37:32.469781354 +0000 UTC m=+67.987461517"
Sep 10 17:37:32.566013 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.565983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:32.631292 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.631241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:32.714505 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.714420    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:32.726305 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.724388    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:37:32.727881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.727860    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.731812 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.731792    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 17:37:32.732711 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.732689    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 17:37:32.732931 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.732915    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 17:37:32.733435 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733406    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 17:37:32.733532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733433    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 17:37:32.733532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733443    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kqxsc\""
Sep 10 17:37:32.733633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733531    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fjfs4gcsldlc5\""
Sep 10 17:37:32.733633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733469    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 17:37:32.733743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733682    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 17:37:32.733743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733692    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 17:37:32.733843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733788    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 17:37:32.733843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.733802    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 17:37:32.734737 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.734717    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 17:37:32.734910 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.734893    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 17:37:32.740627 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.737529    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 17:37:32.740627 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.740134    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 17:37:32.754418 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.754398    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:37:32.799949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.799929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.799965    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.799992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800018    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800044    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800158    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800176    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9fs\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800372    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800417    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800506 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.800879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.800546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901665 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9fs\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901702    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901734    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901785    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.901815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901813    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901839    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901873    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901929    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.901978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902049    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902081    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.902427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.905433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.902656    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.905433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.903286    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.905433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.903529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.906786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.905838    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.906786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.906441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.907390 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.907368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.907850 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.907824    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.909399 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.908371    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.914597 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.914572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9fs\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.918006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.917988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.918189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.918144    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.918794 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.918750    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.918988 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.918961    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.919053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.918999    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.919237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.919193    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.919316 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.919297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.920057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.919561    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.920057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.919774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.920057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.919811    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:32.921988 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:32.921965    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:33.041146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:33.041066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:33.044405 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:33.044381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:33.141817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:33.141746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:33.219328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:33.219300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:35.115621 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.115589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:35.216231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.216162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:35.370098 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.370072    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:37:35.726185 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.726112    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:37:35.756377 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.756346    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:37:35.760507 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.760488    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.770716 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.770696    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:37:35.788767 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:35.788748    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4e1dd9_308e_40f2_b7bf_d9447632ec8c.slice/crio-2b1d86fb9bc8fd8765b7bbf921b29fff52b2e5500a20a6ecd13a83f7bf6646e1 WatchSource:0}: Error finding container 2b1d86fb9bc8fd8765b7bbf921b29fff52b2e5500a20a6ecd13a83f7bf6646e1: Status 404 returned error can't find the container with id 2b1d86fb9bc8fd8765b7bbf921b29fff52b2e5500a20a6ecd13a83f7bf6646e1
Sep 10 17:37:35.832964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.832943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833054 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833054 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833051    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbh9s\" (UniqueName: \"kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833150    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833192    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.833298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.833220    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.933966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934149    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbh9s\" (UniqueName: \"kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.934413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.934267    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.936474 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.936275    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.938183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.936904    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.938183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.937410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.938183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.938116    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.939154 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.938905    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.939268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.939156    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:35.946003 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:35.945979    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbh9s\" (UniqueName: \"kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s\") pod \"console-84c4c89c5b-gtmrh\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") " pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:36.072083 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.072061    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:36.363388 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.363242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:37:36.365995 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:36.365969    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0376f920_b2f6_4b1f_87b3_3c34393513da.slice/crio-9922c83486f688eefd4ccb4dd3c800f0a296d0b40723e684830395829969c6a9 WatchSource:0}: Error finding container 9922c83486f688eefd4ccb4dd3c800f0a296d0b40723e684830395829969c6a9: Status 404 returned error can't find the container with id 9922c83486f688eefd4ccb4dd3c800f0a296d0b40723e684830395829969c6a9
Sep 10 17:37:36.474049 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.474018    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs" event={"ID":"ce5a5c11-d93e-4ca5-8f16-0393e2246481","Type":"ContainerStarted","Data":"fab7beed69060e23f05a6665d9e462cde1b9fd8b2be8f8fc9ecc92592a00edb4"}
Sep 10 17:37:36.475790 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.475766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pj2gf" event={"ID":"eec0661f-f759-4d48-9dcf-1d1934d210d3","Type":"ContainerStarted","Data":"e99cdfc1f734fd98a0a818047821e51007f0111bcb393b7fa16efc566e5000e0"}
Sep 10 17:37:36.475909 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.475885    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:37:36.477488 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.477444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs" event={"ID":"0fe4d4d9-cbea-4857-a3f0-5faacd4c0564","Type":"ContainerStarted","Data":"0ab1433b5389eb69170d30e2980cfda23306f6ffe9a473fa92bbef7b84af1482"}
Sep 10 17:37:36.477695 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.477671    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:36.479200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.479176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c4c89c5b-gtmrh" event={"ID":"0376f920-b2f6-4b1f-87b3-3c34393513da","Type":"ContainerStarted","Data":"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"}
Sep 10 17:37:36.479324 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.479221    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c4c89c5b-gtmrh" event={"ID":"0376f920-b2f6-4b1f-87b3-3c34393513da","Type":"ContainerStarted","Data":"9922c83486f688eefd4ccb4dd3c800f0a296d0b40723e684830395829969c6a9"}
Sep 10 17:37:36.481747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.481725    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6" exitCode=0
Sep 10 17:37:36.481859 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.481821    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"}
Sep 10 17:37:36.481923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.481886    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"2b1d86fb9bc8fd8765b7bbf921b29fff52b2e5500a20a6ecd13a83f7bf6646e1"}
Sep 10 17:37:36.483817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.483788    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs"
Sep 10 17:37:36.485883 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.485863    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"}
Sep 10 17:37:36.485982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.485916    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"}
Sep 10 17:37:36.485982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.485950    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"}
Sep 10 17:37:36.485982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.485980    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"}
Sep 10 17:37:36.486099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.486012    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"}
Sep 10 17:37:36.487337 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.487318    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-79kh2" event={"ID":"b00454b7-c84c-4280-87ca-39149012189f","Type":"ContainerStarted","Data":"1bfd41c6ea91050b27850399179f74e949c77bf583ac52d1bbccb13cade099de"}
Sep 10 17:37:36.495112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.495059    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs" podStartSLOduration=2.427547686 podStartE2EDuration="6.495041249s" totalImagesPullingTime="4.067493563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.494070114 +0000 UTC m=+72.011750277" watchObservedRunningTime="2026-09-10 17:37:36.495041249 +0000 UTC m=+72.012721414"
Sep 10 17:37:36.516243 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.516195    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-jjjcs" podStartSLOduration=2.05481577 podStartE2EDuration="5.516179006s" totalImagesPullingTime="3.461363236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.515355029 +0000 UTC m=+72.033035191" watchObservedRunningTime="2026-09-10 17:37:36.516179006 +0000 UTC m=+72.033859168"
Sep 10 17:37:36.536424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.536385    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84c4c89c5b-gtmrh" podStartSLOduration=1.536371453 podStartE2EDuration="1.536371453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.535752453 +0000 UTC m=+72.053432618" watchObservedRunningTime="2026-09-10 17:37:36.536371453 +0000 UTC m=+72.054051688"
Sep 10 17:37:36.554176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.554139    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pj2gf" podStartSLOduration=65.791653511 podStartE2EDuration="1m11.554125675s" totalImagesPullingTime="5.762472164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.552513844 +0000 UTC m=+72.070194002" watchObservedRunningTime="2026-09-10 17:37:36.554125675 +0000 UTC m=+72.071805844"
Sep 10 17:37:36.569191 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:36.569149    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-79kh2" podStartSLOduration=66.065334241 podStartE2EDuration="1m11.569132015s" totalImagesPullingTime="5.503797774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 17:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:36.56870699 +0000 UTC m=+72.086387154" watchObservedRunningTime="2026-09-10 17:37:36.569132015 +0000 UTC m=+72.086812179"
Sep 10 17:37:37.138778 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.138753    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:37:37.141888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.141861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.177517 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.177490    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:37:37.180411 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.180388    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.197811 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.197757    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:37:37.239605 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.239577    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.244734 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.244836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244752    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w7g9\" (UniqueName: \"kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.244836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244780    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.244945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.244945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.245039 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.244946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.245075 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.245033    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.340558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.340532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.345662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345868 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345878    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w7g9\" (UniqueName: \"kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345975 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.345975 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.345958    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.346611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.346565    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.346709 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.346685    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.346812 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.346793    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.346858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.346808    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.348703 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.348671    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.348843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.348819    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.358608 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.358586    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w7g9\" (UniqueName: \"kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9\") pod \"console-86d6b5b864-8wk5x\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") " pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.440648 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.440571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:37.491565 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.491544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:37.496846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.496816    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerStarted","Data":"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"}
Sep 10 17:37:37.525139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.525085    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.119418615 podStartE2EDuration="10.525070697s" totalImagesPullingTime="7.387098838s" totalInitContainerRuntime="18.553244ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:37.523618958 +0000 UTC m=+73.041299124" watchObservedRunningTime="2026-09-10 17:37:37.525070697 +0000 UTC m=+73.042750879"
Sep 10 17:37:37.885595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:37.885498    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:37:37.887548 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:37:37.887519    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e113b6_567e_419f_89ab_358305ef264d.slice/crio-2b6f31e48fbba7eda133f339f6df6881e50ae093428fdd02d17efb5e76db557b WatchSource:0}: Error finding container 2b6f31e48fbba7eda133f339f6df6881e50ae093428fdd02d17efb5e76db557b: Status 404 returned error can't find the container with id 2b6f31e48fbba7eda133f339f6df6881e50ae093428fdd02d17efb5e76db557b
Sep 10 17:37:38.249817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.249747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:38.350135 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.350103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:38.412310 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.412280    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:37:38.503414 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.503330    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d6b5b864-8wk5x" event={"ID":"87e113b6-567e-419f-89ab-358305ef264d","Type":"ContainerStarted","Data":"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"}
Sep 10 17:37:38.503414 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.503391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d6b5b864-8wk5x" event={"ID":"87e113b6-567e-419f-89ab-358305ef264d","Type":"ContainerStarted","Data":"2b6f31e48fbba7eda133f339f6df6881e50ae093428fdd02d17efb5e76db557b"}
Sep 10 17:37:38.522585 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.522530    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d6b5b864-8wk5x" podStartSLOduration=1.522509832 podStartE2EDuration="1.522509832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:38.520290431 +0000 UTC m=+74.037970594" watchObservedRunningTime="2026-09-10 17:37:38.522509832 +0000 UTC m=+74.040189998"
Sep 10 17:37:38.732344 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:38.732311    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:40.513928 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:40.513895    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"}
Sep 10 17:37:40.514321 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:40.513982    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"}
Sep 10 17:37:41.243231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:41.243184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:41.343564 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:41.343524    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:42.527654 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:42.527613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"}
Sep 10 17:37:42.528088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:42.527686    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"}
Sep 10 17:37:42.528088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:42.527715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"}
Sep 10 17:37:42.528088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:42.527742    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerStarted","Data":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"}
Sep 10 17:37:42.564469 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:42.564397    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.449179877 podStartE2EDuration="10.564374352s" totalImagesPullingTime="5.089128075s" totalInitContainerRuntime="26.0664ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:37:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:37:42.563218867 +0000 UTC m=+78.080899031" watchObservedRunningTime="2026-09-10 17:37:42.564374352 +0000 UTC m=+78.082054515"
Sep 10 17:37:43.045203 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:43.045172    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:37:46.073349 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:46.073231    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:37:47.245773 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.245738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:47.310259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.310232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:47.492282 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.492238    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:47.492531 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.492307    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:47.497494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.497402    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:47.553732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.553710    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:37:47.597403 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.597370    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:37:47.779554 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.779469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:47.885208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:47.885176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:51.252238 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:51.252207    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:51.252642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:51.252258    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:37:52.781657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:52.781617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:37:52.845068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:37:52.845038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:00.748882 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:00.748807    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5985d64575-wbf8p" podUID="ac6db0b1-5287-45af-a8ac-2151402910c3" containerName="console" containerID="cri-o://64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb" gracePeriod=15
Sep 10 17:38:00.982996 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:00.982976    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:38:01.135524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135500    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135554    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4tw4\" (UniqueName: \"kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135578    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135718    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135801    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135830    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.135949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.135864    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert\") pod \"ac6db0b1-5287-45af-a8ac-2151402910c3\" (UID: \"ac6db0b1-5287-45af-a8ac-2151402910c3\") "
Sep 10 17:38:01.136148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.136110    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:01.136570 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.136426    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:01.136677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.136637    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:01.136906 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.136879    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config" (OuterVolumeSpecName: "console-config") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:01.138041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.138006    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:01.138265 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.138123    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:01.138265 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.138122    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4" (OuterVolumeSpecName: "kube-api-access-s4tw4") pod "ac6db0b1-5287-45af-a8ac-2151402910c3" (UID: "ac6db0b1-5287-45af-a8ac-2151402910c3"). InnerVolumeSpecName "kube-api-access-s4tw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:01.237029 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237004    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4tw4\" (UniqueName: \"kubernetes.io/projected/ac6db0b1-5287-45af-a8ac-2151402910c3-kube-api-access-s4tw4\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237029 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237026    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237035    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237045    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237052    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237069    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac6db0b1-5287-45af-a8ac-2151402910c3-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.237143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.237078    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac6db0b1-5287-45af-a8ac-2151402910c3-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:01.605832 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.605763    2571 generic.go:356] "Generic (PLEG): container finished" podID="ac6db0b1-5287-45af-a8ac-2151402910c3" containerID="64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb" exitCode=0
Sep 10 17:38:01.605832 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.605818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5985d64575-wbf8p" event={"ID":"ac6db0b1-5287-45af-a8ac-2151402910c3","Type":"ContainerDied","Data":"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"}
Sep 10 17:38:01.605968 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.605873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5985d64575-wbf8p" event={"ID":"ac6db0b1-5287-45af-a8ac-2151402910c3","Type":"ContainerDied","Data":"27f45b305a9c7e157fd87ba0b5b82cd3bb3c503835b18eed602605fca7f37604"}
Sep 10 17:38:01.605968 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.605898    2571 scope.go:117] "RemoveContainer" containerID="64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"
Sep 10 17:38:01.605968 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.605826    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5985d64575-wbf8p"
Sep 10 17:38:01.615635 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.615608    2571 scope.go:117] "RemoveContainer" containerID="64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"
Sep 10 17:38:01.615970 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:01.615947    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb\": container with ID starting with 64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb not found: ID does not exist" containerID="64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"
Sep 10 17:38:01.616021 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.615980    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb"} err="failed to get container status \"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb\": rpc error: code = NotFound desc = could not find container \"64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb\": container with ID starting with 64646911e955b1234a006a16a19816aa2dc28f09a951c7f335ae1b661af001fb not found: ID does not exist"
Sep 10 17:38:01.625208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.625177    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:38:01.627565 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:01.627542    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5985d64575-wbf8p"]
Sep 10 17:38:03.006971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.006934    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac6db0b1-5287-45af-a8ac-2151402910c3" path="/var/lib/kubelet/pods/ac6db0b1-5287-45af-a8ac-2151402910c3/volumes"
Sep 10 17:38:03.526874 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.526825    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84c4c89c5b-gtmrh" podUID="0376f920-b2f6-4b1f-87b3-3c34393513da" containerName="console" containerID="cri-o://65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b" gracePeriod=15
Sep 10 17:38:03.759528 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.759504    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:38:03.855802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855748    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbh9s\" (UniqueName: \"kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.855918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855823    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.855918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855862    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.855918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855897    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.856025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855927    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.856025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855955    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.856025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.855985    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca\") pod \"0376f920-b2f6-4b1f-87b3-3c34393513da\" (UID: \"0376f920-b2f6-4b1f-87b3-3c34393513da\") "
Sep 10 17:38:03.856394 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.856316    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:03.856526 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.856417    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:03.856584 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.856558    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca" (OuterVolumeSpecName: "service-ca") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:03.856641 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.856606    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config" (OuterVolumeSpecName: "console-config") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:03.858053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.858023    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s" (OuterVolumeSpecName: "kube-api-access-jbh9s") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "kube-api-access-jbh9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:03.858170 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.858104    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:03.858231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.858187    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0376f920-b2f6-4b1f-87b3-3c34393513da" (UID: "0376f920-b2f6-4b1f-87b3-3c34393513da"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:03.957236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957207    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957234    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957248    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957260    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbh9s\" (UniqueName: \"kubernetes.io/projected/0376f920-b2f6-4b1f-87b3-3c34393513da-kube-api-access-jbh9s\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957272    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957284    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0376f920-b2f6-4b1f-87b3-3c34393513da-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:03.957372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:03.957296    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0376f920-b2f6-4b1f-87b3-3c34393513da-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:04.620109 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.620075    2571 generic.go:356] "Generic (PLEG): container finished" podID="0376f920-b2f6-4b1f-87b3-3c34393513da" containerID="65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b" exitCode=0
Sep 10 17:38:04.620539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.620133    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c4c89c5b-gtmrh"
Sep 10 17:38:04.620539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.620175    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c4c89c5b-gtmrh" event={"ID":"0376f920-b2f6-4b1f-87b3-3c34393513da","Type":"ContainerDied","Data":"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"}
Sep 10 17:38:04.620539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.620233    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c4c89c5b-gtmrh" event={"ID":"0376f920-b2f6-4b1f-87b3-3c34393513da","Type":"ContainerDied","Data":"9922c83486f688eefd4ccb4dd3c800f0a296d0b40723e684830395829969c6a9"}
Sep 10 17:38:04.620539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.620257    2571 scope.go:117] "RemoveContainer" containerID="65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"
Sep 10 17:38:04.630363 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.630336    2571 scope.go:117] "RemoveContainer" containerID="65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"
Sep 10 17:38:04.630699 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:04.630679    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b\": container with ID starting with 65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b not found: ID does not exist" containerID="65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"
Sep 10 17:38:04.630768 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.630707    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b"} err="failed to get container status \"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b\": rpc error: code = NotFound desc = could not find container \"65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b\": container with ID starting with 65542407a2d1e165f5a44d5dca24b45454cb78fdbaf51fa3df733a640fcdfe3b not found: ID does not exist"
Sep 10 17:38:04.640442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.640419    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:38:04.646396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:04.646370    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84c4c89c5b-gtmrh"]
Sep 10 17:38:05.006242 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:05.006173    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0376f920-b2f6-4b1f-87b3-3c34393513da" path="/var/lib/kubelet/pods/0376f920-b2f6-4b1f-87b3-3c34393513da/volumes"
Sep 10 17:38:07.499994 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:07.499964    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pj2gf"
Sep 10 17:38:07.784637 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:07.784572    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:07.841051 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:07.841024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:11.259983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:11.259954    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:38:11.264728 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:11.264695    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c9d989b66-nmpgs"
Sep 10 17:38:12.617797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.617726    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cf967bc96-qzf7p" podUID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" containerName="console" containerID="cri-o://6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9" gracePeriod=15
Sep 10 17:38:12.892528 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.892475    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:38:12.931295 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931270    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931465 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931309    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931465 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931360    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6xb4\" (UniqueName: \"kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931465 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931392    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931631 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931479    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931631 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931540    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config\") pod \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\" (UID: \"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f\") "
Sep 10 17:38:12.931736 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931690    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca" (OuterVolumeSpecName: "service-ca") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:12.931800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931790    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:12.931853 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.931783    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config" (OuterVolumeSpecName: "console-config") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:12.932059 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.932033    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:12.933772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.933743    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4" (OuterVolumeSpecName: "kube-api-access-s6xb4") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "kube-api-access-s6xb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:12.933888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.933868    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:12.934057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:12.934040    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" (UID: "5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:13.032649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.032625    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:13.032649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.032647    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:13.032800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.032657    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:13.032800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.032673    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6xb4\" (UniqueName: \"kubernetes.io/projected/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-kube-api-access-s6xb4\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:13.032800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.032682    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:13.672148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.672116    2571 generic.go:356] "Generic (PLEG): container finished" podID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" containerID="6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9" exitCode=0
Sep 10 17:38:13.672636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.672168    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf967bc96-qzf7p"
Sep 10 17:38:13.672636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.672208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf967bc96-qzf7p" event={"ID":"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f","Type":"ContainerDied","Data":"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"}
Sep 10 17:38:13.672636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.672272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf967bc96-qzf7p" event={"ID":"5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f","Type":"ContainerDied","Data":"6999eb2b0f960dd07a13926e3cbbee2e0004397a52b71eaf8a399fb336445137"}
Sep 10 17:38:13.672636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.672295    2571 scope.go:117] "RemoveContainer" containerID="6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"
Sep 10 17:38:13.681277 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.681260    2571 scope.go:117] "RemoveContainer" containerID="6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"
Sep 10 17:38:13.681616 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:13.681596    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9\": container with ID starting with 6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9 not found: ID does not exist" containerID="6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"
Sep 10 17:38:13.681687 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.681630    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9"} err="failed to get container status \"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9\": rpc error: code = NotFound desc = could not find container \"6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9\": container with ID starting with 6868003bc80847557f6320ecf075345b133beba9d76bb6241c8939eff9e500a9 not found: ID does not exist"
Sep 10 17:38:13.687287 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.687266    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:38:13.690366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:13.690343    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cf967bc96-qzf7p"]
Sep 10 17:38:15.006726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:15.006693    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" path="/var/lib/kubelet/pods/5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f/volumes"
Sep 10 17:38:19.982688 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:19.982647    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:20.037207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:20.037174    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:33.045726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:33.045689    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:33.066814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:33.066781    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:33.768043 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:33.768016    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:42.643239 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:42.643201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:42.703743 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:42.703706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:46.882655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.882619    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:46.883148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.882944    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="alertmanager" containerID="cri-o://950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92" gracePeriod=120
Sep 10 17:38:46.883148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.883030    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-web" containerID="cri-o://7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a" gracePeriod=120
Sep 10 17:38:46.883148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.883088    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="config-reloader" containerID="cri-o://b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77" gracePeriod=120
Sep 10 17:38:46.883148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.883116    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="prom-label-proxy" containerID="cri-o://7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9" gracePeriod=120
Sep 10 17:38:46.883148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.883027    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-metric" containerID="cri-o://5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4" gracePeriod=120
Sep 10 17:38:46.883420 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:46.883088    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy" containerID="cri-o://93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68" gracePeriod=120
Sep 10 17:38:47.804838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804804    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9" exitCode=0
Sep 10 17:38:47.804838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804831    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4" exitCode=0
Sep 10 17:38:47.804838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804837    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68" exitCode=0
Sep 10 17:38:47.804838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804845    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77" exitCode=0
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804855    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92" exitCode=0
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"}
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804948    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"}
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.804977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"}
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.805002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"}
Sep 10 17:38:47.805100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:47.805027    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"}
Sep 10 17:38:48.129347 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.129326    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.223106 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223080    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223128    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223155    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223182    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223209    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qm9x\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223231    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223264    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223302    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223346    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223382    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223407    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223445    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.223744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223499    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"79e1877a-2551-4d05-95a8-69d014bcb3f4\" (UID: \"79e1877a-2551-4d05-95a8-69d014bcb3f4\") "
Sep 10 17:38:48.224156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.223976    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:48.224782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.224749    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:48.224898 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.224774    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:48.226566 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.226527    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out" (OuterVolumeSpecName: "config-out") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:48.226759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.226720    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.227088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.227048    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x" (OuterVolumeSpecName: "kube-api-access-9qm9x") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "kube-api-access-9qm9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:48.227493 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.227445    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume" (OuterVolumeSpecName: "config-volume") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.227606 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.227582    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.227696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.227680    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.228008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.227993    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:48.228158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.228142    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.231010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.230985    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.237633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.237433    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config" (OuterVolumeSpecName: "web-config") pod "79e1877a-2551-4d05-95a8-69d014bcb3f4" (UID: "79e1877a-2551-4d05-95a8-69d014bcb3f4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:48.324706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324658    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qm9x\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-kube-api-access-9qm9x\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324699    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-main-db\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324714    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324727    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/79e1877a-2551-4d05-95a8-69d014bcb3f4-tls-assets\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324739    2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-cluster-tls-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324750    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-metrics-client-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324763    2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79e1877a-2551-4d05-95a8-69d014bcb3f4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324775    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324787    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324799    2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-secret-alertmanager-main-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324836    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-web-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324848    2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-volume\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.324971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.324859    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/79e1877a-2551-4d05-95a8-69d014bcb3f4-config-out\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:48.812294 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.812259    2571 generic.go:356] "Generic (PLEG): container finished" podID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerID="7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a" exitCode=0
Sep 10 17:38:48.812480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.812334    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 17:38:48.812480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.812336    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"}
Sep 10 17:38:48.812480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.812405    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"79e1877a-2551-4d05-95a8-69d014bcb3f4","Type":"ContainerDied","Data":"0db38774ed37162466371722d02740c8b757e0e95119ab031ac43f10e55ada8a"}
Sep 10 17:38:48.813406 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.812439    2571 scope.go:117] "RemoveContainer" containerID="7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"
Sep 10 17:38:48.821639 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.821621    2571 scope.go:117] "RemoveContainer" containerID="5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"
Sep 10 17:38:48.828399 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.828383    2571 scope.go:117] "RemoveContainer" containerID="93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"
Sep 10 17:38:48.835421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.835394    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:48.836179 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.836166    2571 scope.go:117] "RemoveContainer" containerID="7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"
Sep 10 17:38:48.838998 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.838977    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 17:38:48.843855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.843831    2571 scope.go:117] "RemoveContainer" containerID="b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"
Sep 10 17:38:48.850370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.850339    2571 scope.go:117] "RemoveContainer" containerID="950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"
Sep 10 17:38:48.857584 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.857561    2571 scope.go:117] "RemoveContainer" containerID="aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc"
Sep 10 17:38:48.864651 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.864634    2571 scope.go:117] "RemoveContainer" containerID="7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"
Sep 10 17:38:48.865041 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.865000    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9\": container with ID starting with 7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9 not found: ID does not exist" containerID="7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"
Sep 10 17:38:48.865143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865051    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9"} err="failed to get container status \"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9\": rpc error: code = NotFound desc = could not find container \"7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9\": container with ID starting with 7968b63f3d0e3db0b3bdf20e3759b9c39545f02fd01e00262ffef055ba7ee6a9 not found: ID does not exist"
Sep 10 17:38:48.865143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865077    2571 scope.go:117] "RemoveContainer" containerID="5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"
Sep 10 17:38:48.865381 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.865360    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4\": container with ID starting with 5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4 not found: ID does not exist" containerID="5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"
Sep 10 17:38:48.865444 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865389    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4"} err="failed to get container status \"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4\": rpc error: code = NotFound desc = could not find container \"5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4\": container with ID starting with 5f3c1964b6f23788e6d8ace954cfc97a5cc66eaebf2318799ee4885323dcb8f4 not found: ID does not exist"
Sep 10 17:38:48.865444 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865409    2571 scope.go:117] "RemoveContainer" containerID="93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"
Sep 10 17:38:48.865720 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.865703    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68\": container with ID starting with 93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68 not found: ID does not exist" containerID="93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"
Sep 10 17:38:48.865784 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865725    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68"} err="failed to get container status \"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68\": rpc error: code = NotFound desc = could not find container \"93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68\": container with ID starting with 93c423cc8c9498811becdb33e356090cacda3c48c486cd6e326529ede8bbbf68 not found: ID does not exist"
Sep 10 17:38:48.865784 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.865746    2571 scope.go:117] "RemoveContainer" containerID="7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"
Sep 10 17:38:48.865997 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.865980    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a\": container with ID starting with 7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a not found: ID does not exist" containerID="7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"
Sep 10 17:38:48.866058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866005    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a"} err="failed to get container status \"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a\": rpc error: code = NotFound desc = could not find container \"7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a\": container with ID starting with 7a08a0f7731756e8cdd0acc6d3910788de3d890851a71db8c0db031349613a4a not found: ID does not exist"
Sep 10 17:38:48.866058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866027    2571 scope.go:117] "RemoveContainer" containerID="b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"
Sep 10 17:38:48.866267 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.866252    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77\": container with ID starting with b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77 not found: ID does not exist" containerID="b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"
Sep 10 17:38:48.866331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866270    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77"} err="failed to get container status \"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77\": rpc error: code = NotFound desc = could not find container \"b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77\": container with ID starting with b98252866fbd1ba80e6247c17cc154c6ace2ce927f7efc0a0c034dcf80d5fb77 not found: ID does not exist"
Sep 10 17:38:48.866331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866283    2571 scope.go:117] "RemoveContainer" containerID="950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"
Sep 10 17:38:48.866566 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.866549    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92\": container with ID starting with 950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92 not found: ID does not exist" containerID="950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"
Sep 10 17:38:48.866636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866577    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92"} err="failed to get container status \"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92\": rpc error: code = NotFound desc = could not find container \"950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92\": container with ID starting with 950f6f8075aabed348aee995bc6ebf61b0930d53b4665dab5c0d8e91121a8f92 not found: ID does not exist"
Sep 10 17:38:48.866636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866594    2571 scope.go:117] "RemoveContainer" containerID="aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc"
Sep 10 17:38:48.866844 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:48.866821    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc\": container with ID starting with aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc not found: ID does not exist" containerID="aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc"
Sep 10 17:38:48.866888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:48.866851    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc"} err="failed to get container status \"aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc\": rpc error: code = NotFound desc = could not find container \"aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc\": container with ID starting with aef268df77cdf823c3ee305a3e3cae77c52e62afe5e3c5fbfe56efd124ee1abc not found: ID does not exist"
Sep 10 17:38:49.006674 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:49.006649    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" path="/var/lib/kubelet/pods/79e1877a-2551-4d05-95a8-69d014bcb3f4/volumes"
Sep 10 17:38:51.093851 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.093816    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:51.094284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094126    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="prometheus" containerID="cri-o://aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500" gracePeriod=600
Sep 10 17:38:51.094284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094168    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="thanos-sidecar" containerID="cri-o://8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c" gracePeriod=600
Sep 10 17:38:51.094284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094189    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="config-reloader" containerID="cri-o://808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102" gracePeriod=600
Sep 10 17:38:51.094284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094204    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b" gracePeriod=600
Sep 10 17:38:51.094284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094148    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy" containerID="cri-o://feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0" gracePeriod=600
Sep 10 17:38:51.094604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.094227    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-web" containerID="cri-o://1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43" gracePeriod=600
Sep 10 17:38:51.331809 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.331785    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:51.449397 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449365    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449397 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449403    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449427    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449476    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449603    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449649    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449681    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm9fs\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449731    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449760    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.449795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449785    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449823    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449853    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449898    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449906    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449928    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449961    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.449991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.450016    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.450077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.450009    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.450530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.450433    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.450965 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.450764    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.451232 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451203    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.451329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451255    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.451329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451290    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.451329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451313    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets\") pod \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\" (UID: \"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c\") "
Sep 10 17:38:51.451799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451575    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-metrics-client-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.451799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451597    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.451799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451614    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.451799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451629    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.451799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.451641    2571 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.452640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.452602    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.452761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.452729    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.452851 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.452829    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.452913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.452827    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs" (OuterVolumeSpecName: "kube-api-access-xm9fs") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "kube-api-access-xm9fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:51.452913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.452899    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.453475 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.453168    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.453475 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.453177    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.453936 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.453874    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out" (OuterVolumeSpecName: "config-out") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:51.454079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.454054    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:38:51.454152 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.454105    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:38:51.454308 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.454294    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:38:51.455242 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.455217    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.455374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.455350    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.455420 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.455406    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config" (OuterVolumeSpecName: "config") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.463987 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.463968    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config" (OuterVolumeSpecName: "web-config") pod "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" (UID: "aa4e1dd9-308e-40f2-b7bf-d9447632ec8c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:38:51.552738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552695    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552734    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552745    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552754    2571 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-kube-rbac-proxy\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552763    2571 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552772    2571 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-prometheus-k8s-db\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552780    2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-web-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552788    2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-tls-assets\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552795    2571 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552803    2571 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-grpc-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552811    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552821    2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-config-out\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552830    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm9fs\" (UniqueName: \"kubernetes.io/projected/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-kube-api-access-xm9fs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552838    2571 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-metrics-client-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.552989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.552846    2571 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830914    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830942    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830948    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830954    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830959    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830965    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500" exitCode=0
Sep 10 17:38:51.830995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830992    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.830991    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831059    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831086    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831111    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831127    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831135    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"}
Sep 10 17:38:51.831300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.831183    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa4e1dd9-308e-40f2-b7bf-d9447632ec8c","Type":"ContainerDied","Data":"2b1d86fb9bc8fd8765b7bbf921b29fff52b2e5500a20a6ecd13a83f7bf6646e1"}
Sep 10 17:38:51.839492 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.839478    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.846215 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.846199    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.852863 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.852845    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.855667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.855648    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:51.859770 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.859755    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.860160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.860141    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:51.866025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.866009    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.873019 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.873003    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.879554 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.879537    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.879835 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.879819    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.879889 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.879844    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.879889 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.879861    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.880121 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.880103    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.880186 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880129    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.880186 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880153    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.880390 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.880375    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.880427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880394    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.880427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880406    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.880688 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.880660    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.880782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880700    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.880782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880714    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.880981 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.880964    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.881038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880984    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.881038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.880996    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.881388 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.881358    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.881446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881391    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.881446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881405    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.881709 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:38:51.881689    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.881788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881712    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.881788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881726    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.881981 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881965    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.882018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.881982    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.882256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882236    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.882298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882257    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.882542 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882524    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.882608 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882544    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.882814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882795    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.882868 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.882818    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.883063 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883044    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.883155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883064    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.883276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883257    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.883336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883278    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.883531 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883513    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.883583 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883532    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.883744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883727    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.883814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883745    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.883956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883937    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.884022 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.883957    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.884183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884166    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.884228 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884185    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.884409 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884392    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.884492 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884411    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.884649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884633    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.884716 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884650    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.884892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884875    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.884950 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.884893    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.885115 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885098    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.885158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885118    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.885315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885299    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.885360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885315    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.885555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885538    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.885616 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885556    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.885790 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885774    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.885832 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885790    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.886005 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.885989    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.886048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886006    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.886204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886188    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.886276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886205    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.886413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886396    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.886511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886414    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.886631 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886614    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.886672 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886634    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.886858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886842    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.886923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.886859    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.887082 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887065    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.887126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887083    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.887306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887290    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.887345 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887309    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.887550 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887534    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.887622 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887552    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.887770 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887754    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.887835 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887772    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.887993 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887977    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.888036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.887994    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.888210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888195    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.888270 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888212    2571 scope.go:117] "RemoveContainer" containerID="8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"
Sep 10 17:38:51.888448 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888424    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b"} err="failed to get container status \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": rpc error: code = NotFound desc = could not find container \"8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b\": container with ID starting with 8545dcf329d72c0aa8b9e9c35d51614e0f33e27f78530dea56c17cceb8dd3a0b not found: ID does not exist"
Sep 10 17:38:51.888526 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888448    2571 scope.go:117] "RemoveContainer" containerID="feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"
Sep 10 17:38:51.888828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888770    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0"} err="failed to get container status \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": rpc error: code = NotFound desc = could not find container \"feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0\": container with ID starting with feef214934d7955864135f4fd9f27799de9723d16d54b74ff3653e09f06709c0 not found: ID does not exist"
Sep 10 17:38:51.888828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.888816    2571 scope.go:117] "RemoveContainer" containerID="1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"
Sep 10 17:38:51.889172 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889149    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43"} err="failed to get container status \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": rpc error: code = NotFound desc = could not find container \"1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43\": container with ID starting with 1c64209b1cdf3626c5acda4e023127116402355bafe243d640ed85c8bd0fab43 not found: ID does not exist"
Sep 10 17:38:51.889242 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889174    2571 scope.go:117] "RemoveContainer" containerID="8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"
Sep 10 17:38:51.889530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889442    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c"} err="failed to get container status \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": rpc error: code = NotFound desc = could not find container \"8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c\": container with ID starting with 8761d8d8177c0e67e289505e797c408c88cf2361a5dfa9f437eb90d49dbb926c not found: ID does not exist"
Sep 10 17:38:51.889530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889484    2571 scope.go:117] "RemoveContainer" containerID="808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"
Sep 10 17:38:51.889763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889742    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102"} err="failed to get container status \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": rpc error: code = NotFound desc = could not find container \"808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102\": container with ID starting with 808f97fc3820a1a757c49121a95b11deeabcccdce27e24b28a253db059c5b102 not found: ID does not exist"
Sep 10 17:38:51.889763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.889763    2571 scope.go:117] "RemoveContainer" containerID="aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"
Sep 10 17:38:51.890062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890039    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500"} err="failed to get container status \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": rpc error: code = NotFound desc = could not find container \"aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500\": container with ID starting with aeee9d71aff26404b893450480a6efcdbb163951fd7f28b3d0467a096598a500 not found: ID does not exist"
Sep 10 17:38:51.890062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890063    2571 scope.go:117] "RemoveContainer" containerID="83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"
Sep 10 17:38:51.890190 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890072    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:51.890301 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890283    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6"} err="failed to get container status \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": rpc error: code = NotFound desc = could not find container \"83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6\": container with ID starting with 83e490d08c5c0e8b5ebe41563029be7cb10786e9482115bd3883613c98799cf6 not found: ID does not exist"
Sep 10 17:38:51.890346 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890326    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="config-reloader"
Sep 10 17:38:51.890346 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890338    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="config-reloader"
Sep 10 17:38:51.890346 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890346    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890351    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890359    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-thanos"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890364    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-thanos"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890371    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="prom-label-proxy"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890377    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="prom-label-proxy"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890383    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="alertmanager"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890388    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="alertmanager"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890393    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="prometheus"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890398    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="prometheus"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890404    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890410    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890417    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0376f920-b2f6-4b1f-87b3-3c34393513da" containerName="console"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890422    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0376f920-b2f6-4b1f-87b3-3c34393513da" containerName="console"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890428    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac6db0b1-5287-45af-a8ac-2151402910c3" containerName="console"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890432    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac6db0b1-5287-45af-a8ac-2151402910c3" containerName="console"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890440    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="thanos-sidecar"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890444    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="thanos-sidecar"
Sep 10 17:38:51.890446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890468    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="init-config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890474    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="init-config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890479    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890484    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890489    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" containerName="console"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890495    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" containerName="console"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890500    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-metric"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890504    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-metric"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890510    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890514    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890520    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="init-config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890524    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="init-config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890529    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890533    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890569    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-thanos"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890576    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890581    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890585    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy-metric"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890591    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="prom-label-proxy"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890596    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy-web"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890602    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0376f920-b2f6-4b1f-87b3-3c34393513da" containerName="console"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890607    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="kube-rbac-proxy"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890611    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="thanos-sidecar"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890616    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac6db0b1-5287-45af-a8ac-2151402910c3" containerName="console"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890620    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5097fae6-74c8-4aea-bbe7-9d6e6bfa4f7f" containerName="console"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890625    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="kube-rbac-proxy"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890629    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="alertmanager"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890633    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79e1877a-2551-4d05-95a8-69d014bcb3f4" containerName="config-reloader"
Sep 10 17:38:51.891093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.890637    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" containerName="prometheus"
Sep 10 17:38:51.894173 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.894157    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:51.897701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.897194    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kqxsc\""
Sep 10 17:38:51.897701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.897218    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 17:38:51.897701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.897345    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 17:38:51.897701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.897661    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.898547    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fjfs4gcsldlc5\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.898716    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899061    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899139    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899221    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899361    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899587    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.899862    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.900032    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 17:38:51.902443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.900265    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 17:38:51.904347 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.904329    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 17:38:51.905097 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.905078    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 17:38:51.912364 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:51.912339    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:52.056330 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056282    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056330 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056356    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056383    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056421    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056437    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056560    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056627    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056669    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056692    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056708    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v7gq\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-kube-api-access-8v7gq\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.056813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056801    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.057036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056829    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.057036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056849    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.057036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.056863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158202    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158248    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158392    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158431    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158522    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.158696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.158669    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159279 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159250    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159363 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159274    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159363 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159363 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v7gq\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-kube-api-access-8v7gq\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159534 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159360    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159534 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159378    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159847 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159824    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159931 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.159931 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159913    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.159980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.160013    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160132 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.160042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160132 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.160071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160132 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.160100    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.160273 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.160183    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.161560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.161435    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.161560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.161487    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.161560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.161548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.162502 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.161701    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163147 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.162996    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163294 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163641    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163643    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163646    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.163908 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.163867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.164692 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.164668    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.165111 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.165093    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.165797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.165774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.167803 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.167784    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v7gq\" (UniqueName: \"kubernetes.io/projected/f6dff5b7-7056-4a0d-9cce-ed0d6360b193-kube-api-access-8v7gq\") pod \"prometheus-k8s-0\" (UID: \"f6dff5b7-7056-4a0d-9cce-ed0d6360b193\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.207734 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.207712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:38:52.489191 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.489161    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 17:38:52.492321 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:38:52.492288    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dff5b7_7056_4a0d_9cce_ed0d6360b193.slice/crio-e132893ff0ab34c5d513a36f7569ce54d59e2af7d717716534e06732037b0689 WatchSource:0}: Error finding container e132893ff0ab34c5d513a36f7569ce54d59e2af7d717716534e06732037b0689: Status 404 returned error can't find the container with id e132893ff0ab34c5d513a36f7569ce54d59e2af7d717716534e06732037b0689
Sep 10 17:38:52.839777 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.839685    2571 generic.go:356] "Generic (PLEG): container finished" podID="f6dff5b7-7056-4a0d-9cce-ed0d6360b193" containerID="1210054d81b1a7cd9036967b664c65f535584eea04e7e5facca464a9d7f35a7e" exitCode=0
Sep 10 17:38:52.839945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.839772    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerDied","Data":"1210054d81b1a7cd9036967b664c65f535584eea04e7e5facca464a9d7f35a7e"}
Sep 10 17:38:52.839945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:52.839863    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"e132893ff0ab34c5d513a36f7569ce54d59e2af7d717716534e06732037b0689"}
Sep 10 17:38:53.009489 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.009117    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4e1dd9-308e-40f2-b7bf-d9447632ec8c" path="/var/lib/kubelet/pods/aa4e1dd9-308e-40f2-b7bf-d9447632ec8c/volumes"
Sep 10 17:38:53.847917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.847885    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"8860dd6cde531eb18a56503267acceb1d182cc17b37e1dbd7eb78541a9951554"}
Sep 10 17:38:53.848293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.847952    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"68691e440ff6560ea2d918fc48c885469914e73676566d5ad0139df27e46705e"}
Sep 10 17:38:53.848293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.847978    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"b8353b50889f05489c9d3fe5de73718a765098af4ecda5001047437ea16b5bd0"}
Sep 10 17:38:53.848293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.848001    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"74f21ddf791d9b14392ff637a86ff93314a6df03514169cc5ed0b9c4301be03e"}
Sep 10 17:38:53.848293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.848027    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"2b86a744e9f41f7a23b380a09c3cd5e6e037a3c429648f1c7b9318edab604403"}
Sep 10 17:38:53.848293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.848049    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6dff5b7-7056-4a0d-9cce-ed0d6360b193","Type":"ContainerStarted","Data":"c876061d5932d0e378ca2687fd5f428a8c0178b8f6f77e39681eab07aecb3903"}
Sep 10 17:38:53.878750 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:53.878700    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.859319708 podStartE2EDuration="2.878685184s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.365476ms" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:53.876605093 +0000 UTC m=+149.394285258" watchObservedRunningTime="2026-09-10 17:38:53.878685184 +0000 UTC m=+149.396365380"
Sep 10 17:38:54.421785 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.421753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:54.486973 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.486942    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:54.759343 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.759271    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:38:54.761593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.761579    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.779336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.779316    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:38:54.782875 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.782852    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.782963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.782886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.782963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.782949    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.783067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.782973    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.783067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.783004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfv6p\" (UniqueName: \"kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.783156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.783068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.783156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.783113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.821640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.821618    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:54.884095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884369    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884508    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884768 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884768 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884661    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfv6p\" (UniqueName: \"kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.884909 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.884885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.885476 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.885433    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.885681 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.885664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.885782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.885762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.887231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.887214    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.887379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.887361    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:54.889870 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.889847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:38:54.892689 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:54.892660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfv6p\" (UniqueName: \"kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p\") pod \"console-7c4dc5c774-7ldw2\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") " pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:55.069864 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:55.069795    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:38:55.337949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:55.337877    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:38:55.340807 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:38:55.340781    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3314c390_d09b_4357_ad9a_e9dbd257a094.slice/crio-ac4ea20afa667194dead34ec85cb587a4303b3d214f31fa30adc446ca87f5f53 WatchSource:0}: Error finding container ac4ea20afa667194dead34ec85cb587a4303b3d214f31fa30adc446ca87f5f53: Status 404 returned error can't find the container with id ac4ea20afa667194dead34ec85cb587a4303b3d214f31fa30adc446ca87f5f53
Sep 10 17:38:55.859531 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:55.859492    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4dc5c774-7ldw2" event={"ID":"3314c390-d09b-4357-ad9a-e9dbd257a094","Type":"ContainerStarted","Data":"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"}
Sep 10 17:38:55.859708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:55.859567    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4dc5c774-7ldw2" event={"ID":"3314c390-d09b-4357-ad9a-e9dbd257a094","Type":"ContainerStarted","Data":"ac4ea20afa667194dead34ec85cb587a4303b3d214f31fa30adc446ca87f5f53"}
Sep 10 17:38:55.878419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:55.878364    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c4dc5c774-7ldw2" podStartSLOduration=1.878345704 podStartE2EDuration="1.878345704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:38:55.87787451 +0000 UTC m=+151.395554696" watchObservedRunningTime="2026-09-10 17:38:55.878345704 +0000 UTC m=+151.396025867"
Sep 10 17:38:57.208726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:38:57.208696    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:39:05.070300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:05.070268    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:39:05.070675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:05.070315    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:39:05.074956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:05.074936    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:39:05.899711 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:05.899682    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:39:05.945382 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:05.945354    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:39:06.057843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:06.057813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:06.158890 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:06.158786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:06.216288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:06.216260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:07.797333 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:07.797301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:07.895304 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:07.895275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:25.097790 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:25.097753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:25.197606 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:25.197531    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:29.271360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:29.271328    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:29.372079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:29.372051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:30.970093 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:30.970048    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86d6b5b864-8wk5x" podUID="87e113b6-567e-419f-89ab-358305ef264d" containerName="console" containerID="cri-o://6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918" gracePeriod=15
Sep 10 17:39:31.210551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.210528    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:39:31.362653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362625    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362685    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362714    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362753    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362768    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w7g9\" (UniqueName: \"kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.362795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.362787    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config\") pod \"87e113b6-567e-419f-89ab-358305ef264d\" (UID: \"87e113b6-567e-419f-89ab-358305ef264d\") "
Sep 10 17:39:31.363206 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.363088    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:31.363370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.363213    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config" (OuterVolumeSpecName: "console-config") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:31.363370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.363238    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca" (OuterVolumeSpecName: "service-ca") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:31.363370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.363263    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:39:31.365164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.365139    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:31.365273 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.365253    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:39:31.365334 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.365267    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9" (OuterVolumeSpecName: "kube-api-access-6w7g9") pod "87e113b6-567e-419f-89ab-358305ef264d" (UID: "87e113b6-567e-419f-89ab-358305ef264d"). InnerVolumeSpecName "kube-api-access-6w7g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:39:31.464073 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464049    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464073 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464069    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w7g9\" (UniqueName: \"kubernetes.io/projected/87e113b6-567e-419f-89ab-358305ef264d-kube-api-access-6w7g9\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464080    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464088    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464097    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464105    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/87e113b6-567e-419f-89ab-358305ef264d-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.464221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.464112    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/87e113b6-567e-419f-89ab-358305ef264d-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:39:31.989739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.989706    2571 generic.go:356] "Generic (PLEG): container finished" podID="87e113b6-567e-419f-89ab-358305ef264d" containerID="6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918" exitCode=0
Sep 10 17:39:31.990130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.989750    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d6b5b864-8wk5x"
Sep 10 17:39:31.990130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.989758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d6b5b864-8wk5x" event={"ID":"87e113b6-567e-419f-89ab-358305ef264d","Type":"ContainerDied","Data":"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"}
Sep 10 17:39:31.990130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.989798    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d6b5b864-8wk5x" event={"ID":"87e113b6-567e-419f-89ab-358305ef264d","Type":"ContainerDied","Data":"2b6f31e48fbba7eda133f339f6df6881e50ae093428fdd02d17efb5e76db557b"}
Sep 10 17:39:31.990130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.989818    2571 scope.go:117] "RemoveContainer" containerID="6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"
Sep 10 17:39:31.998660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.998642    2571 scope.go:117] "RemoveContainer" containerID="6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"
Sep 10 17:39:31.998926 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:39:31.998906    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918\": container with ID starting with 6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918 not found: ID does not exist" containerID="6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"
Sep 10 17:39:31.998990 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:31.998937    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918"} err="failed to get container status \"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918\": rpc error: code = NotFound desc = could not find container \"6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918\": container with ID starting with 6f8b67f84231d5254223e280c1734c41320de0b2aaf16bc6e440b24a16ebb918 not found: ID does not exist"
Sep 10 17:39:32.009877 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:32.009858    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:39:32.011619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:32.011595    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86d6b5b864-8wk5x"]
Sep 10 17:39:33.005582 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:33.005553    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87e113b6-567e-419f-89ab-358305ef264d" path="/var/lib/kubelet/pods/87e113b6-567e-419f-89ab-358305ef264d/volumes"
Sep 10 17:39:51.821996 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:51.821961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:52.208829 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:52.208785    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:39:52.224562 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:52.224535    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:39:52.600225 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:52.600134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:53.077420 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:53.077394    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 17:39:55.433575 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:55.433545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:55.538029 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:55.538004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:57.219699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:57.219658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:57.325076 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:57.325029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:59.545603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:59.545571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:39:59.737291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:39:59.737257    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:00.154334 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.154305    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"]
Sep 10 17:40:00.154587 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.154576    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e113b6-567e-419f-89ab-358305ef264d" containerName="console"
Sep 10 17:40:00.154638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.154589    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e113b6-567e-419f-89ab-358305ef264d" containerName="console"
Sep 10 17:40:00.154638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.154633    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87e113b6-567e-419f-89ab-358305ef264d" containerName="console"
Sep 10 17:40:00.156818 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.156803    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.159239 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.159213    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ntjbj\""
Sep 10 17:40:00.159364 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.159242    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 17:40:00.160270 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.160251    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 17:40:00.166201 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.166183    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"]
Sep 10 17:40:00.267399 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.267367    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.267547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.267403    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-664zd\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-kube-api-access-664zd\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.368741 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.368702    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.368741 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.368743    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-664zd\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-kube-api-access-664zd\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.377295 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.377263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.377438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.377420    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-664zd\" (UniqueName: \"kubernetes.io/projected/5045a966-5cf3-4eaf-9ee3-5430a65be32a-kube-api-access-664zd\") pod \"cert-manager-cainjector-559cdd8c87-flnpn\" (UID: \"5045a966-5cf3-4eaf-9ee3-5430a65be32a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.485593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.485523    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"
Sep 10 17:40:00.752235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:00.752167    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-flnpn"]
Sep 10 17:40:00.754995 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:40:00.754969    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5045a966_5cf3_4eaf_9ee3_5430a65be32a.slice/crio-ec1d4f598aa7f8506084d2d0dd3b9d0e00d9c17a095e53bb6ba23bd7da79adb7 WatchSource:0}: Error finding container ec1d4f598aa7f8506084d2d0dd3b9d0e00d9c17a095e53bb6ba23bd7da79adb7: Status 404 returned error can't find the container with id ec1d4f598aa7f8506084d2d0dd3b9d0e00d9c17a095e53bb6ba23bd7da79adb7
Sep 10 17:40:01.088866 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:01.088791    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn" event={"ID":"5045a966-5cf3-4eaf-9ee3-5430a65be32a","Type":"ContainerStarted","Data":"ec1d4f598aa7f8506084d2d0dd3b9d0e00d9c17a095e53bb6ba23bd7da79adb7"}
Sep 10 17:40:02.226020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:02.225983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:02.289748 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:02.289662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:04.103328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:04.103294    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn" event={"ID":"5045a966-5cf3-4eaf-9ee3-5430a65be32a","Type":"ContainerStarted","Data":"1f456df1a0f73a81d0a123c4d0d77635da8aff06351e8df2997f2f8cc5bee422"}
Sep 10 17:40:04.125775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:04.125727    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-flnpn" podStartSLOduration=0.898868335 podStartE2EDuration="4.125706259s" totalImagesPullingTime="3.226837924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:04.124679697 +0000 UTC m=+219.642359858" watchObservedRunningTime="2026-09-10 17:40:04.125706259 +0000 UTC m=+219.643386421"
Sep 10 17:40:07.533924 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:07.533892    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:08.026678 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:08.026642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:08.131198 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:08.131163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:09.337628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:09.337594    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:09.438758 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:09.438729    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.673504 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:13.673469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:13.775655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:13.775625    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:15.558938 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:15.558904    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:18.145020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:18.144986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:20.233784 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:20.233748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:20.290976 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:20.290934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.874710 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:32.874675    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:32.965341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:32.965309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:33.443771 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.443738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:33.731046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.730968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:33.842887 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.842858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:33.861625 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.861600    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"]
Sep 10 17:40:33.865737 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.865720    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:33.868565 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.868541    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c9dnv\""
Sep 10 17:40:33.868807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.868774    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 17:40:33.868807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.868798    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 17:40:33.868953 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.868779    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 17:40:33.869431 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.869415    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 17:40:33.883963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.883944    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"]
Sep 10 17:40:33.911783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.911762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:33.911872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.911791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:33.911872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.911812    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:33.911941 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.911910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:33.911941 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:33.911935    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczh8\" (UniqueName: \"kubernetes.io/projected/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-kube-api-access-hczh8\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.009413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.009333    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.012385 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.012464 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hczh8\" (UniqueName: \"kubernetes.io/projected/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-kube-api-access-hczh8\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.012543 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012526    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.012591 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.012726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.012774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.012741    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-manifests\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.014884 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.014854    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-tmp\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.015065 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.015044    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.015154 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.015132    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.021584 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.021561    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczh8\" (UniqueName: \"kubernetes.io/projected/c6b2dca6-2edb-4cc1-a39a-22426bf01dd4-kube-api-access-hczh8\") pod \"opendatahub-operator-controller-manager-dbf885bd4-m897n\" (UID: \"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4\") " pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.106967 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.106938    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.176056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.176031    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:34.204626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.204596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.304342 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.304256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.416776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.416720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:34.452048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:34.452011    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"]
Sep 10 17:40:34.453400 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:40:34.453377    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b2dca6_2edb_4cc1_a39a_22426bf01dd4.slice/crio-1fb2db89a18dac6d20104e88e7941cd517d45d5c8c5c806c36afb32aff9f7197 WatchSource:0}: Error finding container 1fb2db89a18dac6d20104e88e7941cd517d45d5c8c5c806c36afb32aff9f7197: Status 404 returned error can't find the container with id 1fb2db89a18dac6d20104e88e7941cd517d45d5c8c5c806c36afb32aff9f7197
Sep 10 17:40:35.223023 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:35.222964    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n" event={"ID":"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4","Type":"ContainerStarted","Data":"1fb2db89a18dac6d20104e88e7941cd517d45d5c8c5c806c36afb32aff9f7197"}
Sep 10 17:40:38.237288 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:38.237245    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n" event={"ID":"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4","Type":"ContainerStarted","Data":"b00538edf9b121cfeedef8ee741c8360e16fc492903980a248f5edae4eb40b59"}
Sep 10 17:40:39.242545 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:39.242507    2571 generic.go:356] "Generic (PLEG): container finished" podID="c6b2dca6-2edb-4cc1-a39a-22426bf01dd4" containerID="b00538edf9b121cfeedef8ee741c8360e16fc492903980a248f5edae4eb40b59" exitCode=0
Sep 10 17:40:39.243011 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:39.242570    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n" event={"ID":"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4","Type":"ContainerDied","Data":"b00538edf9b121cfeedef8ee741c8360e16fc492903980a248f5edae4eb40b59"}
Sep 10 17:40:39.927266 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:39.927228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:40.013729 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:40.013693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:40.247807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:40.247726    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n" event={"ID":"c6b2dca6-2edb-4cc1-a39a-22426bf01dd4","Type":"ContainerStarted","Data":"085590dbf3ac413fbc2e99e35f676e9c032e7cfefc7f691cc68dc89354a1eab5"}
Sep 10 17:40:40.248175 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:40.247863    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:40.273187 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:40.273132    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n" podStartSLOduration=2.8993407749999998 podStartE2EDuration="7.273114406s" totalImagesPullingTime="3.358313204s" totalInitContainerRuntime="1.015460427s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:40.271086302 +0000 UTC m=+255.788766465" watchObservedRunningTime="2026-09-10 17:40:40.273114406 +0000 UTC m=+255.790794566"
Sep 10 17:40:45.394688 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:45.394650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:45.495158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:45.495126    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:51.253969 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:51.253937    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbf885bd4-m897n"
Sep 10 17:40:52.840714 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:52.840681    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:53.027342 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.027311    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:53.247802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.247766    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:53.403581 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.403545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:53.426310 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.426284    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"]
Sep 10 17:40:53.429369 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.429355    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.434679 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.434645    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 17:40:53.434679 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.434667    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 17:40:53.434852 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.434667    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n5lq4\""
Sep 10 17:40:53.434852 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.434719    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 17:40:53.434852 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.434746    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 17:40:53.444249 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.444230    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"]
Sep 10 17:40:53.554293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.554219    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/807feed5-8bc2-4b14-82ab-0bfa85face7b-tmp\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.554293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.554265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjbv\" (UniqueName: \"kubernetes.io/projected/807feed5-8bc2-4b14-82ab-0bfa85face7b-kube-api-access-7pjbv\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.554477 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.554300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/807feed5-8bc2-4b14-82ab-0bfa85face7b-tls-certs\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.655496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.655442    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/807feed5-8bc2-4b14-82ab-0bfa85face7b-tmp\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.655588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.655519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjbv\" (UniqueName: \"kubernetes.io/projected/807feed5-8bc2-4b14-82ab-0bfa85face7b-kube-api-access-7pjbv\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.655588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.655547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/807feed5-8bc2-4b14-82ab-0bfa85face7b-tls-certs\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.657895 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.657858    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/807feed5-8bc2-4b14-82ab-0bfa85face7b-tmp\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.658069 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.658053    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/807feed5-8bc2-4b14-82ab-0bfa85face7b-tls-certs\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.663129 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.663108    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjbv\" (UniqueName: \"kubernetes.io/projected/807feed5-8bc2-4b14-82ab-0bfa85face7b-kube-api-access-7pjbv\") pod \"kube-auth-proxy-7cd5576685-bstbn\" (UID: \"807feed5-8bc2-4b14-82ab-0bfa85face7b\") " pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:53.738060 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:53.738032    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"
Sep 10 17:40:54.008856 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.008825    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cd5576685-bstbn"]
Sep 10 17:40:54.011626 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:40:54.011602    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807feed5_8bc2_4b14_82ab_0bfa85face7b.slice/crio-a62afd9864f481418881f9872cc781bc3432fc3655e67f38d56ce8993f477e11 WatchSource:0}: Error finding container a62afd9864f481418881f9872cc781bc3432fc3655e67f38d56ce8993f477e11: Status 404 returned error can't find the container with id a62afd9864f481418881f9872cc781bc3432fc3655e67f38d56ce8993f477e11
Sep 10 17:40:54.298811 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.298742    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn" event={"ID":"807feed5-8bc2-4b14-82ab-0bfa85face7b","Type":"ContainerStarted","Data":"a62afd9864f481418881f9872cc781bc3432fc3655e67f38d56ce8993f477e11"}
Sep 10 17:40:54.699930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.699897    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:40:54.703802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.703602    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.703937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.703815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:54.706422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.706402    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 17:40:54.706761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.706570    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tn85x\""
Sep 10 17:40:54.706761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.706641    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 17:40:54.706927 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.706822    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 17:40:54.706927 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.706839    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 17:40:54.717440 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.717418    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:40:54.772691 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.772661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:54.865149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865061    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865202    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5n7t\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.865423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.865383    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5n7t\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966477    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.966626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.966502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.967840 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.967790    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.970196 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.970167    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.970791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.970763    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.971150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.971109    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.971325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.971301    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.976041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.976019    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:54.977338 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:54.977293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5n7t\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t\") pod \"istiod-openshift-gateway-7b7995d5cd-9zbws\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:55.016523 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:55.016447    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:55.306179 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:55.306096    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:40:55.309573 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:40:55.309543    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1409ac8_0b06_4753_aa47_6a63d40f91f4.slice/crio-f7557ef7625ce6dcf03fbe01ebd41bd7bc3d1cd8382ef4b14a76b2097a967b4f WatchSource:0}: Error finding container f7557ef7625ce6dcf03fbe01ebd41bd7bc3d1cd8382ef4b14a76b2097a967b4f: Status 404 returned error can't find the container with id f7557ef7625ce6dcf03fbe01ebd41bd7bc3d1cd8382ef4b14a76b2097a967b4f
Sep 10 17:40:55.746296 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:55.746259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:56.310905 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:56.310870    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" event={"ID":"c1409ac8-0b06-4753-aa47-6a63d40f91f4","Type":"ContainerStarted","Data":"f7557ef7625ce6dcf03fbe01ebd41bd7bc3d1cd8382ef4b14a76b2097a967b4f"}
Sep 10 17:40:57.316885 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:57.316840    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn" event={"ID":"807feed5-8bc2-4b14-82ab-0bfa85face7b","Type":"ContainerStarted","Data":"ed59733871cf1b1d33d884b5a96e67d877697f2df6edc048b8f1acd5566c7470"}
Sep 10 17:40:57.334820 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:57.334763    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cd5576685-bstbn" podStartSLOduration=1.523009171 podStartE2EDuration="4.334742217s" totalImagesPullingTime="2.811733046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:57.332629703 +0000 UTC m=+272.850309891" watchObservedRunningTime="2026-09-10 17:40:57.334742217 +0000 UTC m=+272.852422381"
Sep 10 17:40:58.566466 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:58.566405    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 17:40:58.566746 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:58.566509    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 17:40:59.246614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.246574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:40:59.325955 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.325922    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" event={"ID":"c1409ac8-0b06-4753-aa47-6a63d40f91f4","Type":"ContainerStarted","Data":"dd6cccd43dd29ca070e030abac9be6cecd10361ce5e6edb6e1b1ecbdfffff6c0"}
Sep 10 17:40:59.326110 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.326080    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:40:59.327689 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.327660    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-9zbws container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 17:40:59.327823 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.327725    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 17:40:59.375138 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:40:59.375089    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" podStartSLOduration=2.121252423 podStartE2EDuration="5.37507557s" totalImagesPullingTime="3.253823147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:40:59.373916854 +0000 UTC m=+274.891597053" watchObservedRunningTime="2026-09-10 17:40:59.37507557 +0000 UTC m=+274.892755732"
Sep 10 17:41:00.331072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:00.331046    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:41:00.495930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:00.495897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:03.941337 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:03.941308    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:04.045493 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:04.045442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:04.107720 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:04.107691    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:04.375149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:04.375099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:04.482572 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:04.482543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:11.728144 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.728111    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"]
Sep 10 17:41:11.735291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.735270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:11.737930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.737903    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tkbtv\""
Sep 10 17:41:11.738908 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.738878    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 17:41:11.739008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.738926    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 17:41:11.739008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.738931    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 17:41:11.739008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.738943    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 17:41:11.739008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.738968    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 17:41:11.747470 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.746207    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"]
Sep 10 17:41:11.899684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.899646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxxnt\" (UniqueName: \"kubernetes.io/projected/860ddcbd-d13e-439a-bd1f-e1392d1fd340-kube-api-access-bxxnt\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:11.899825 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.899717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:11.899825 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.899740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/860ddcbd-d13e-439a-bd1f-e1392d1fd340-manager-config\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:11.899904 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:11.899839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-metrics-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.001028 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.000957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.001028 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.000987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/860ddcbd-d13e-439a-bd1f-e1392d1fd340-manager-config\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.001028 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.001022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-metrics-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.001325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.001055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxxnt\" (UniqueName: \"kubernetes.io/projected/860ddcbd-d13e-439a-bd1f-e1392d1fd340-kube-api-access-bxxnt\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.001635 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.001607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/860ddcbd-d13e-439a-bd1f-e1392d1fd340-manager-config\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.003636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.003607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-metrics-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.003724 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.003670    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/860ddcbd-d13e-439a-bd1f-e1392d1fd340-cert\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.010998 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.010978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxxnt\" (UniqueName: \"kubernetes.io/projected/860ddcbd-d13e-439a-bd1f-e1392d1fd340-kube-api-access-bxxnt\") pod \"lws-controller-manager-57bb875478-5s8mk\" (UID: \"860ddcbd-d13e-439a-bd1f-e1392d1fd340\") " pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.047870 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.047852    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:12.325046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.325011    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"]
Sep 10 17:41:12.327339 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:41:12.327312    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860ddcbd_d13e_439a_bd1f_e1392d1fd340.slice/crio-9a8503224dc7dc5d7303ba4b8ab8e43737a0b251c32bcf17caa45396f990f340 WatchSource:0}: Error finding container 9a8503224dc7dc5d7303ba4b8ab8e43737a0b251c32bcf17caa45396f990f340: Status 404 returned error can't find the container with id 9a8503224dc7dc5d7303ba4b8ab8e43737a0b251c32bcf17caa45396f990f340
Sep 10 17:41:12.373780 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:12.373751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk" event={"ID":"860ddcbd-d13e-439a-bd1f-e1392d1fd340","Type":"ContainerStarted","Data":"9a8503224dc7dc5d7303ba4b8ab8e43737a0b251c32bcf17caa45396f990f340"}
Sep 10 17:41:13.779494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:13.779437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:13.885802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:13.885765    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:14.066256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:14.066159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:14.224052 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:14.224003    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:14.324184 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:14.324087    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:15.227032 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.227000    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:41:15.227407 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.227190    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery" containerID="cri-o://dd6cccd43dd29ca070e030abac9be6cecd10361ce5e6edb6e1b1ecbdfffff6c0" gracePeriod=30
Sep 10 17:41:15.331430 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.331402    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-9zbws container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.23:8080/ready\": dial tcp 10.134.0.23:8080: connect: connection refused" start-of-body=
Sep 10 17:41:15.331574 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.331492    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.23:8080/ready\": dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 10 17:41:15.390400 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.390373    2571 generic.go:356] "Generic (PLEG): container finished" podID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerID="dd6cccd43dd29ca070e030abac9be6cecd10361ce5e6edb6e1b1ecbdfffff6c0" exitCode=0
Sep 10 17:41:15.390539 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.390414    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" event={"ID":"c1409ac8-0b06-4753-aa47-6a63d40f91f4","Type":"ContainerDied","Data":"dd6cccd43dd29ca070e030abac9be6cecd10361ce5e6edb6e1b1ecbdfffff6c0"}
Sep 10 17:41:15.392438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.392407    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk" event={"ID":"860ddcbd-d13e-439a-bd1f-e1392d1fd340","Type":"ContainerStarted","Data":"fcd72e803937e153a493c8fc23ecb10434b4b66d9ea719ca23b778f8ddeb7801"}
Sep 10 17:41:15.392583 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.392507    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:41:15.431363 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.431320    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk" podStartSLOduration=1.968040214 podStartE2EDuration="4.43130639s" totalImagesPullingTime="2.463266176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:41:15.430669508 +0000 UTC m=+290.948349671" watchObservedRunningTime="2026-09-10 17:41:15.43130639 +0000 UTC m=+290.948986552"
Sep 10 17:41:15.454742 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.454722    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:41:15.530603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530540    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530572    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530598    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530824 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530640    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530824 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530670    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530824 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530709    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.530824 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530733    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5n7t\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t\") pod \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\" (UID: \"c1409ac8-0b06-4753-aa47-6a63d40f91f4\") "
Sep 10 17:41:15.531071 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.530958    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:41:15.533031 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533001    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:41:15.533139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533086    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts" (OuterVolumeSpecName: "cacerts") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:41:15.533238 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533215    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:41:15.533297 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533264    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:41:15.533374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533355    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs" (OuterVolumeSpecName: "local-certs") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:41:15.533507 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.533484    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t" (OuterVolumeSpecName: "kube-api-access-t5n7t") pod "c1409ac8-0b06-4753-aa47-6a63d40f91f4" (UID: "c1409ac8-0b06-4753-aa47-6a63d40f91f4"). InnerVolumeSpecName "kube-api-access-t5n7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:41:15.631816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631783    2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1409ac8-0b06-4753-aa47-6a63d40f91f4-local-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.631816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631809    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-ca-configmap\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.631816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631819    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-csr-dns-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.632048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631829    2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-kubeconfig\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.632048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631837    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-istio-token\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.632048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631845    2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1409ac8-0b06-4753-aa47-6a63d40f91f4-cacerts\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:15.632048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:15.631854    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5n7t\" (UniqueName: \"kubernetes.io/projected/c1409ac8-0b06-4753-aa47-6a63d40f91f4-kube-api-access-t5n7t\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:41:16.397775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:16.397743    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"
Sep 10 17:41:16.398184 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:16.397771    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws" event={"ID":"c1409ac8-0b06-4753-aa47-6a63d40f91f4","Type":"ContainerDied","Data":"f7557ef7625ce6dcf03fbe01ebd41bd7bc3d1cd8382ef4b14a76b2097a967b4f"}
Sep 10 17:41:16.398184 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:16.397824    2571 scope.go:117] "RemoveContainer" containerID="dd6cccd43dd29ca070e030abac9be6cecd10361ce5e6edb6e1b1ecbdfffff6c0"
Sep 10 17:41:16.419522 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:16.419491    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:41:16.427655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:16.427630    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9zbws"]
Sep 10 17:41:17.006604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:17.006575    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" path="/var/lib/kubelet/pods/c1409ac8-0b06-4753-aa47-6a63d40f91f4/volumes"
Sep 10 17:41:23.259178 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:23.259145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:23.359244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:23.359212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:41:24.880691 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:24.880670    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 17:41:26.400593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:41:26.400563    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57bb875478-5s8mk"
Sep 10 17:42:05.150588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:05.150544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:05.248669 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:05.248627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:17.581479 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:17.581379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:18.386341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:18.386303    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:19.192045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:19.192006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:21.525388 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:21.525355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:21.624119 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:21.624086    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:22.869117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:22.869085    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:23.849785 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:23.849756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:25.012067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.012038    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c69794f4-4m8cg"]
Sep 10 17:42:25.012544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.012350    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery"
Sep 10 17:42:25.012544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.012368    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery"
Sep 10 17:42:25.012544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.012435    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1409ac8-0b06-4753-aa47-6a63d40f91f4" containerName="discovery"
Sep 10 17:42:25.015313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.015287    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.015671 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.015649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:25.034519 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.034496    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c69794f4-4m8cg"]
Sep 10 17:42:25.156293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-oauth-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.156433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89689\" (UniqueName: \"kubernetes.io/projected/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-kube-api-access-89689\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.156748 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156732    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-service-ca\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.156796 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156767    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.156885 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-oauth-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.156982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.156953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-trusted-ca-bundle\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.157279 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.157143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.205423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.205397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:25.258176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.258280 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-oauth-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.258280 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-trusted-ca-bundle\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.258280 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.258432 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-oauth-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.258432 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89689\" (UniqueName: \"kubernetes.io/projected/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-kube-api-access-89689\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.259247 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.258385    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-service-ca\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.259351 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.259273    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-oauth-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.264761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.259563    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-trusted-ca-bundle\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.264761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.260488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-service-ca\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.264761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.261263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.264761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.261652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-serving-cert\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.265841 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.265819    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-console-oauth-config\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.279760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.279733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89689\" (UniqueName: \"kubernetes.io/projected/b8bf0a3a-6e81-490e-aad8-e6ce79315f81-kube-api-access-89689\") pod \"console-7c69794f4-4m8cg\" (UID: \"b8bf0a3a-6e81-490e-aad8-e6ce79315f81\") " pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.324541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.324517    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:25.376080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.376043    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:25.603659 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.603637    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c69794f4-4m8cg"]
Sep 10 17:42:25.606135 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:42:25.606108    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bf0a3a_6e81_490e_aad8_e6ce79315f81.slice/crio-7d43619f87e9af347b9fd9ea2dcf1bc8e6b9a6284314db5a2520188c274cc4d6 WatchSource:0}: Error finding container 7d43619f87e9af347b9fd9ea2dcf1bc8e6b9a6284314db5a2520188c274cc4d6: Status 404 returned error can't find the container with id 7d43619f87e9af347b9fd9ea2dcf1bc8e6b9a6284314db5a2520188c274cc4d6
Sep 10 17:42:25.656193 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:25.656161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c69794f4-4m8cg" event={"ID":"b8bf0a3a-6e81-490e-aad8-e6ce79315f81","Type":"ContainerStarted","Data":"7d43619f87e9af347b9fd9ea2dcf1bc8e6b9a6284314db5a2520188c274cc4d6"}
Sep 10 17:42:26.576309 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.576271    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"]
Sep 10 17:42:26.579431 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.579415    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:26.582622 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.582604    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m6265\""
Sep 10 17:42:26.583494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.583479    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 17:42:26.583545 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.583515    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 17:42:26.592146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.592123    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"]
Sep 10 17:42:26.661712 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.661682    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c69794f4-4m8cg" event={"ID":"b8bf0a3a-6e81-490e-aad8-e6ce79315f81","Type":"ContainerStarted","Data":"4b1f903de9190fd86aed4e5e49945e6c8254622ebf1919d83dd25185218bbe2e"}
Sep 10 17:42:26.670129 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.670105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c559d\" (UniqueName: \"kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d\") pod \"limitador-operator-controller-manager-74495c69f8-758kt\" (UID: \"e772b0d8-b466-41d3-ae93-114c8b28b418\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:26.684879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.684834    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c69794f4-4m8cg" podStartSLOduration=2.684820854 podStartE2EDuration="2.684820854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:26.684004215 +0000 UTC m=+362.201684388" watchObservedRunningTime="2026-09-10 17:42:26.684820854 +0000 UTC m=+362.202501015"
Sep 10 17:42:26.770760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.770728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c559d\" (UniqueName: \"kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d\") pod \"limitador-operator-controller-manager-74495c69f8-758kt\" (UID: \"e772b0d8-b466-41d3-ae93-114c8b28b418\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:26.786433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.786404    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c559d\" (UniqueName: \"kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d\") pod \"limitador-operator-controller-manager-74495c69f8-758kt\" (UID: \"e772b0d8-b466-41d3-ae93-114c8b28b418\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:26.842628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.842560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:26.889657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:26.889628    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:27.164981 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.164957    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"]
Sep 10 17:42:27.167304 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:42:27.167273    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode772b0d8_b466_41d3_ae93_114c8b28b418.slice/crio-fc263fcd0c751f48122cb6e840a38e9983bdee8b50dc26e8ba95da73e007bdc2 WatchSource:0}: Error finding container fc263fcd0c751f48122cb6e840a38e9983bdee8b50dc26e8ba95da73e007bdc2: Status 404 returned error can't find the container with id fc263fcd0c751f48122cb6e840a38e9983bdee8b50dc26e8ba95da73e007bdc2
Sep 10 17:42:27.169477 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.169440    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:42:27.195280 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.195251    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:27.580250 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.580176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:27.667125 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.667093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt" event={"ID":"e772b0d8-b466-41d3-ae93-114c8b28b418","Type":"ContainerStarted","Data":"fc263fcd0c751f48122cb6e840a38e9983bdee8b50dc26e8ba95da73e007bdc2"}
Sep 10 17:42:27.684241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:27.684216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:28.086555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:28.086519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:28.207595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:28.207553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:29.536219 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.536189    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"]
Sep 10 17:42:29.538621 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.538596    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.541644 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.541623    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jrtvh\""
Sep 10 17:42:29.541764 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.541714    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 17:42:29.541982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.541967    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 17:42:29.552659 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.552637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:29.553532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.553515    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"]
Sep 10 17:42:29.591677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.591500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88jph\" (UniqueName: \"kubernetes.io/projected/8e2e0e76-065a-46cd-b9e5-768fa5263b10-kube-api-access-88jph\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.591677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.591541    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2e0e76-065a-46cd-b9e5-768fa5263b10-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.591677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.591607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e2e0e76-065a-46cd-b9e5-768fa5263b10-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.678375 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.678346    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt" event={"ID":"e772b0d8-b466-41d3-ae93-114c8b28b418","Type":"ContainerStarted","Data":"124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329"}
Sep 10 17:42:29.678530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.678431    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:29.692999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.692978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e2e0e76-065a-46cd-b9e5-768fa5263b10-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.693124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.693045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88jph\" (UniqueName: \"kubernetes.io/projected/8e2e0e76-065a-46cd-b9e5-768fa5263b10-kube-api-access-88jph\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.693124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.693070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2e0e76-065a-46cd-b9e5-768fa5263b10-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.693705 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.693684    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8e2e0e76-065a-46cd-b9e5-768fa5263b10-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.695372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.695354    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e2e0e76-065a-46cd-b9e5-768fa5263b10-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.704511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.704490    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88jph\" (UniqueName: \"kubernetes.io/projected/8e2e0e76-065a-46cd-b9e5-768fa5263b10-kube-api-access-88jph\") pod \"kuadrant-console-plugin-fbdd4f445-4z92c\" (UID: \"8e2e0e76-065a-46cd-b9e5-768fa5263b10\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:29.723271 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.723230    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt" podStartSLOduration=1.299871229 podStartE2EDuration="3.723217481s" totalImagesPullingTime="2.423346252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:29.72076822 +0000 UTC m=+365.238448381" watchObservedRunningTime="2026-09-10 17:42:29.723217481 +0000 UTC m=+365.240897643"
Sep 10 17:42:29.768198 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.768180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:29.849190 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:29.849128    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"
Sep 10 17:42:30.122934 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:30.122785    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c"]
Sep 10 17:42:30.125519 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:42:30.125494    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e2e0e76_065a_46cd_b9e5_768fa5263b10.slice/crio-dc3bdc9295ac9b1ce646f3895dc478e7f3e613aab9248e59b6bb4614a266beed WatchSource:0}: Error finding container dc3bdc9295ac9b1ce646f3895dc478e7f3e613aab9248e59b6bb4614a266beed: Status 404 returned error can't find the container with id dc3bdc9295ac9b1ce646f3895dc478e7f3e613aab9248e59b6bb4614a266beed
Sep 10 17:42:30.684009 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:30.683980    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c" event={"ID":"8e2e0e76-065a-46cd-b9e5-768fa5263b10","Type":"ContainerStarted","Data":"dc3bdc9295ac9b1ce646f3895dc478e7f3e613aab9248e59b6bb4614a266beed"}
Sep 10 17:42:32.598917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:32.598886    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:32.666257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:32.666212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:32.999241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:32.999205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:33.099200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:33.099167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:33.395640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:33.395606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:33.495737 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:33.495700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:35.325500 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.325467    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:35.325919 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.325540    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:35.330197 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.330175    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:35.707284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.707251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c" event={"ID":"8e2e0e76-065a-46cd-b9e5-768fa5263b10","Type":"ContainerStarted","Data":"e5d0a97db6d93c4648ad66a913c27a64138b57814484145008ce5ff57beabf6f"}
Sep 10 17:42:35.711104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.711081    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c69794f4-4m8cg"
Sep 10 17:42:35.722751 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.722703    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4z92c" podStartSLOduration=2.200589132 podStartE2EDuration="6.722685363s" totalImagesPullingTime="4.522096231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:35.722537023 +0000 UTC m=+371.240217184" watchObservedRunningTime="2026-09-10 17:42:35.722685363 +0000 UTC m=+371.240365525"
Sep 10 17:42:35.772315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.772291    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:42:35.915910 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:35.915879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:36.016158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:36.016087    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:36.115180 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:36.115155    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:38.903212 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:38.903186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:39.107558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:39.107521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:40.686344 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.686313    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:40.885733 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.885696    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"]
Sep 10 17:42:40.885970 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.885943    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" containerName="manager" containerID="cri-o://124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329" gracePeriod=2
Sep 10 17:42:40.888754 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:42:40.888716    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-758kt\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:40.890417 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.890393    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"]
Sep 10 17:42:40.935431 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.935403    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"]
Sep 10 17:42:40.936614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.936543    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" containerName="manager"
Sep 10 17:42:40.936614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.936569    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" containerName="manager"
Sep 10 17:42:40.936755 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.936676    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" containerName="manager"
Sep 10 17:42:40.939589 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.939570    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:40.954001 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.953979    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"]
Sep 10 17:42:40.978856 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:42:40.978825    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-758kt\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:40.983188 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:40.983167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4sx2\" (UniqueName: \"kubernetes.io/projected/cfce37ba-84e0-45b2-a10b-c7512c1cf106-kube-api-access-r4sx2\") pod \"limitador-operator-controller-manager-74495c69f8-nvcx5\" (UID: \"cfce37ba-84e0-45b2-a10b-c7512c1cf106\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:41.084283 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.084244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4sx2\" (UniqueName: \"kubernetes.io/projected/cfce37ba-84e0-45b2-a10b-c7512c1cf106-kube-api-access-r4sx2\") pod \"limitador-operator-controller-manager-74495c69f8-nvcx5\" (UID: \"cfce37ba-84e0-45b2-a10b-c7512c1cf106\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:41.094657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.094625    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4sx2\" (UniqueName: \"kubernetes.io/projected/cfce37ba-84e0-45b2-a10b-c7512c1cf106-kube-api-access-r4sx2\") pod \"limitador-operator-controller-manager-74495c69f8-nvcx5\" (UID: \"cfce37ba-84e0-45b2-a10b-c7512c1cf106\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:41.124815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.124796    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:41.185636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.185608    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c559d\" (UniqueName: \"kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d\") pod \"e772b0d8-b466-41d3-ae93-114c8b28b418\" (UID: \"e772b0d8-b466-41d3-ae93-114c8b28b418\") "
Sep 10 17:42:41.187773 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.187722    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d" (OuterVolumeSpecName: "kube-api-access-c559d") pod "e772b0d8-b466-41d3-ae93-114c8b28b418" (UID: "e772b0d8-b466-41d3-ae93-114c8b28b418"). InnerVolumeSpecName "kube-api-access-c559d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:42:41.256025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.256004    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:41.286163 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.286138    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c559d\" (UniqueName: \"kubernetes.io/projected/e772b0d8-b466-41d3-ae93-114c8b28b418-kube-api-access-c559d\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:42:41.545398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.545370    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"]
Sep 10 17:42:41.547764 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:42:41.547738    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfce37ba_84e0_45b2_a10b_c7512c1cf106.slice/crio-5008b46c9a7372ecbbba7f6a1499d8a63d462bcb537c19eb9fc0a41fdbe8c6be WatchSource:0}: Error finding container 5008b46c9a7372ecbbba7f6a1499d8a63d462bcb537c19eb9fc0a41fdbe8c6be: Status 404 returned error can't find the container with id 5008b46c9a7372ecbbba7f6a1499d8a63d462bcb537c19eb9fc0a41fdbe8c6be
Sep 10 17:42:41.737879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.737849    2571 generic.go:356] "Generic (PLEG): container finished" podID="e772b0d8-b466-41d3-ae93-114c8b28b418" containerID="124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329" exitCode=0
Sep 10 17:42:41.738279 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.737898    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-758kt"
Sep 10 17:42:41.738279 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.737902    2571 scope.go:117] "RemoveContainer" containerID="124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329"
Sep 10 17:42:41.739745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.739723    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5" event={"ID":"cfce37ba-84e0-45b2-a10b-c7512c1cf106","Type":"ContainerStarted","Data":"c6234a8477e5741cef252a2d3d1e6dce6e81a75ac1ddbf0212660d96f533f115"}
Sep 10 17:42:41.739867 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.739763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5" event={"ID":"cfce37ba-84e0-45b2-a10b-c7512c1cf106","Type":"ContainerStarted","Data":"5008b46c9a7372ecbbba7f6a1499d8a63d462bcb537c19eb9fc0a41fdbe8c6be"}
Sep 10 17:42:41.739935 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.739868    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:41.745912 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.745895    2571 scope.go:117] "RemoveContainer" containerID="124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329"
Sep 10 17:42:41.746173 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:42:41.746152    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329\": container with ID starting with 124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329 not found: ID does not exist" containerID="124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329"
Sep 10 17:42:41.746264 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.746178    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329"} err="failed to get container status \"124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329\": rpc error: code = NotFound desc = could not find container \"124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329\": container with ID starting with 124272f440e8af29ef470fcbee2182f02bff5a26bc029bcb1dca268372f1d329 not found: ID does not exist"
Sep 10 17:42:41.757845 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:41.757792    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5" podStartSLOduration=1.757781402 podStartE2EDuration="1.757781402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:42:41.757536863 +0000 UTC m=+377.275217027" watchObservedRunningTime="2026-09-10 17:42:41.757781402 +0000 UTC m=+377.275461563"
Sep 10 17:42:43.006628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:43.006595    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e772b0d8-b466-41d3-ae93-114c8b28b418" path="/var/lib/kubelet/pods/e772b0d8-b466-41d3-ae93-114c8b28b418/volumes"
Sep 10 17:42:43.760713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:43.760682    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:43.935676 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:43.935650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:48.425953 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:48.425918    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:48.525732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:48.525703    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:52.747255 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:52.747223    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-nvcx5"
Sep 10 17:42:53.630139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:53.630102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:53.729126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:53.729102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:54.050124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:54.050019    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:54.142862 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:54.142826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:54.247237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:54.247208    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:42:54.347237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:42:54.347152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:00.722560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:00.722525    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:00.793182 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:00.793131    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c4dc5c774-7ldw2" podUID="3314c390-d09b-4357-ad9a-e9dbd257a094" containerName="console" containerID="cri-o://1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e" gracePeriod=15
Sep 10 17:43:00.812759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:00.812736    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:01.032061 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.032037    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:43:01.130687 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130651    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130704    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130728    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130834    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfv6p\" (UniqueName: \"kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130865    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.130956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.130939    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert\") pod \"3314c390-d09b-4357-ad9a-e9dbd257a094\" (UID: \"3314c390-d09b-4357-ad9a-e9dbd257a094\") "
Sep 10 17:43:01.131156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131050    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config" (OuterVolumeSpecName: "console-config") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:43:01.131156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131086    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:43:01.131300 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131282    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-console-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.131368 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131307    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-trusted-ca-bundle\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.131368 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131333    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:43:01.131368 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.131331    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca" (OuterVolumeSpecName: "service-ca") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:43:01.133117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.133096    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:43:01.133436 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.133412    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p" (OuterVolumeSpecName: "kube-api-access-tfv6p") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "kube-api-access-tfv6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:43:01.133436 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.133411    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3314c390-d09b-4357-ad9a-e9dbd257a094" (UID: "3314c390-d09b-4357-ad9a-e9dbd257a094"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:43:01.232626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.232586    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfv6p\" (UniqueName: \"kubernetes.io/projected/3314c390-d09b-4357-ad9a-e9dbd257a094-kube-api-access-tfv6p\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.232626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.232624    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.232626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.232635    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-oauth-config\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.232814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.232643    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3314c390-d09b-4357-ad9a-e9dbd257a094-oauth-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.232814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.232653    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3314c390-d09b-4357-ad9a-e9dbd257a094-console-serving-cert\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:01.822912 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.822874    2571 generic.go:356] "Generic (PLEG): container finished" podID="3314c390-d09b-4357-ad9a-e9dbd257a094" containerID="1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e" exitCode=0
Sep 10 17:43:01.823293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.822921    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4dc5c774-7ldw2"
Sep 10 17:43:01.823293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.822956    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4dc5c774-7ldw2" event={"ID":"3314c390-d09b-4357-ad9a-e9dbd257a094","Type":"ContainerDied","Data":"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"}
Sep 10 17:43:01.823293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.823005    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4dc5c774-7ldw2" event={"ID":"3314c390-d09b-4357-ad9a-e9dbd257a094","Type":"ContainerDied","Data":"ac4ea20afa667194dead34ec85cb587a4303b3d214f31fa30adc446ca87f5f53"}
Sep 10 17:43:01.823293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.823025    2571 scope.go:117] "RemoveContainer" containerID="1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"
Sep 10 17:43:01.831657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.831641    2571 scope.go:117] "RemoveContainer" containerID="1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"
Sep 10 17:43:01.831921 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:43:01.831904    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e\": container with ID starting with 1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e not found: ID does not exist" containerID="1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"
Sep 10 17:43:01.831977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.831932    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e"} err="failed to get container status \"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e\": rpc error: code = NotFound desc = could not find container \"1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e\": container with ID starting with 1b89a473c71d6340977815e5ac285600fcb2480e2531cd732e13d04b1ab22c8e not found: ID does not exist"
Sep 10 17:43:01.846302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.846281    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:43:01.852129 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:01.852109    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c4dc5c774-7ldw2"]
Sep 10 17:43:03.006229 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:03.006196    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3314c390-d09b-4357-ad9a-e9dbd257a094" path="/var/lib/kubelet/pods/3314c390-d09b-4357-ad9a-e9dbd257a094/volumes"
Sep 10 17:43:05.445271 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:05.445230    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:05.552971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:05.552930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:06.749644 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:06.749616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:10.231005 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:10.230975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:10.332036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:10.331999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:10.823655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:10.823620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:11.174619 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:11.174595    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:11.275234 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:11.275176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:11.713543 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:11.713481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:12.916980 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:12.916949    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:13.029399 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.029335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:13.141305 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.141281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:13.686042 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.686013    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:43:13.686303 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.686292    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3314c390-d09b-4357-ad9a-e9dbd257a094" containerName="console"
Sep 10 17:43:13.686348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.686305    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3314c390-d09b-4357-ad9a-e9dbd257a094" containerName="console"
Sep 10 17:43:13.686387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.686348    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3314c390-d09b-4357-ad9a-e9dbd257a094" containerName="console"
Sep 10 17:43:13.689110 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.689093    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.695509 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.695349    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 17:43:13.703174 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.703152    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:43:13.777592 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.777563    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:43:13.828233 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.828207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j874\" (UniqueName: \"kubernetes.io/projected/f608ec2a-3953-46e1-a116-79b27d453365-kube-api-access-9j874\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.828355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.828245    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f608ec2a-3953-46e1-a116-79b27d453365-config-file\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.929392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.929361    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j874\" (UniqueName: \"kubernetes.io/projected/f608ec2a-3953-46e1-a116-79b27d453365-kube-api-access-9j874\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.929745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.929413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f608ec2a-3953-46e1-a116-79b27d453365-config-file\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.930017 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.929997    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f608ec2a-3953-46e1-a116-79b27d453365-config-file\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.946431 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.946372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j874\" (UniqueName: \"kubernetes.io/projected/f608ec2a-3953-46e1-a116-79b27d453365-kube-api-access-9j874\") pod \"limitador-limitador-84994f7ffc-7g2kt\" (UID: \"f608ec2a-3953-46e1-a116-79b27d453365\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:13.955405 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.955386    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:13.957596 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.957582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:13.965528 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.965503    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dhx2d\""
Sep 10 17:43:13.979020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.978994    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:13.998938 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:13.998918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:14.021433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.021407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:14.030611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.030590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72g9k\" (UniqueName: \"kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k\") pod \"authorino-7bcfd8f9cd-qztpg\" (UID: \"9e2619ad-434a-4de9-924d-807a32e8541d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:14.125839 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.125811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:14.131989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.131957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72g9k\" (UniqueName: \"kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k\") pod \"authorino-7bcfd8f9cd-qztpg\" (UID: \"9e2619ad-434a-4de9-924d-807a32e8541d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:14.144119 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.144098    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72g9k\" (UniqueName: \"kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k\") pod \"authorino-7bcfd8f9cd-qztpg\" (UID: \"9e2619ad-434a-4de9-924d-807a32e8541d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:14.266826 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.266804    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:14.267709 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.267690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:43:14.269773 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:43:14.269751    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf608ec2a_3953_46e1_a116_79b27d453365.slice/crio-999d85804305975640f9ec62d35b54c120c7fe323243513af03f5a67d7a7ade9 WatchSource:0}: Error finding container 999d85804305975640f9ec62d35b54c120c7fe323243513af03f5a67d7a7ade9: Status 404 returned error can't find the container with id 999d85804305975640f9ec62d35b54c120c7fe323243513af03f5a67d7a7ade9
Sep 10 17:43:14.539772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.539737    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:14.541869 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:43:14.541847    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e2619ad_434a_4de9_924d_807a32e8541d.slice/crio-3b3314909eead7fd7afe004d8ef40d1c7907f222511fd9768db4e999258436f1 WatchSource:0}: Error finding container 3b3314909eead7fd7afe004d8ef40d1c7907f222511fd9768db4e999258436f1: Status 404 returned error can't find the container with id 3b3314909eead7fd7afe004d8ef40d1c7907f222511fd9768db4e999258436f1
Sep 10 17:43:14.879858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.879826    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" event={"ID":"9e2619ad-434a-4de9-924d-807a32e8541d","Type":"ContainerStarted","Data":"3b3314909eead7fd7afe004d8ef40d1c7907f222511fd9768db4e999258436f1"}
Sep 10 17:43:14.881317 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:14.881288    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt" event={"ID":"f608ec2a-3953-46e1-a116-79b27d453365","Type":"ContainerStarted","Data":"999d85804305975640f9ec62d35b54c120c7fe323243513af03f5a67d7a7ade9"}
Sep 10 17:43:15.888701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:15.888664    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt" event={"ID":"f608ec2a-3953-46e1-a116-79b27d453365","Type":"ContainerStarted","Data":"a0d430b5b428c5fb0a60ba28af70169733b79eab41c1ce2bf8f03b6e6fc9e3b1"}
Sep 10 17:43:15.889148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:15.888769    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:15.916437 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:15.916383    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt" podStartSLOduration=2.096886829 podStartE2EDuration="2.916361987s" totalImagesPullingTime="819.475158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:15.915080768 +0000 UTC m=+411.432760928" watchObservedRunningTime="2026-09-10 17:43:15.916361987 +0000 UTC m=+411.434042153"
Sep 10 17:43:16.750774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:16.750739    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:16.894448 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:16.894413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" event={"ID":"9e2619ad-434a-4de9-924d-807a32e8541d","Type":"ContainerStarted","Data":"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"}
Sep 10 17:43:16.910065 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:16.910020    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" podStartSLOduration=2.252973164 podStartE2EDuration="3.910004675s" totalImagesPullingTime="1.657031511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:43:16.909225134 +0000 UTC m=+412.426905298" watchObservedRunningTime="2026-09-10 17:43:16.910004675 +0000 UTC m=+412.427684838"
Sep 10 17:43:17.898326 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:17.898288    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" podUID="9e2619ad-434a-4de9-924d-807a32e8541d" containerName="authorino" containerID="cri-o://da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc" gracePeriod=30
Sep 10 17:43:18.134165 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.134144    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:18.266442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.266367    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72g9k\" (UniqueName: \"kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k\") pod \"9e2619ad-434a-4de9-924d-807a32e8541d\" (UID: \"9e2619ad-434a-4de9-924d-807a32e8541d\") "
Sep 10 17:43:18.268505 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.268472    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k" (OuterVolumeSpecName: "kube-api-access-72g9k") pod "9e2619ad-434a-4de9-924d-807a32e8541d" (UID: "9e2619ad-434a-4de9-924d-807a32e8541d"). InnerVolumeSpecName "kube-api-access-72g9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:43:18.367311 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.367285    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72g9k\" (UniqueName: \"kubernetes.io/projected/9e2619ad-434a-4de9-924d-807a32e8541d-kube-api-access-72g9k\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:18.904160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.904130    2571 generic.go:356] "Generic (PLEG): container finished" podID="9e2619ad-434a-4de9-924d-807a32e8541d" containerID="da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc" exitCode=0
Sep 10 17:43:18.904548 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.904174    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg"
Sep 10 17:43:18.904548 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.904216    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" event={"ID":"9e2619ad-434a-4de9-924d-807a32e8541d","Type":"ContainerDied","Data":"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"}
Sep 10 17:43:18.904548 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.904254    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-qztpg" event={"ID":"9e2619ad-434a-4de9-924d-807a32e8541d","Type":"ContainerDied","Data":"3b3314909eead7fd7afe004d8ef40d1c7907f222511fd9768db4e999258436f1"}
Sep 10 17:43:18.904548 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.904271    2571 scope.go:117] "RemoveContainer" containerID="da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"
Sep 10 17:43:18.912729 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.912712    2571 scope.go:117] "RemoveContainer" containerID="da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"
Sep 10 17:43:18.912985 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:43:18.912968    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc\": container with ID starting with da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc not found: ID does not exist" containerID="da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"
Sep 10 17:43:18.913034 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.912994    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc"} err="failed to get container status \"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc\": rpc error: code = NotFound desc = could not find container \"da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc\": container with ID starting with da5cb2d2fc90616490a1495e139546723db44b215286c942edb7c6cc27bbb9fc not found: ID does not exist"
Sep 10 17:43:18.925998 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.925963    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:18.933415 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:18.933391    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-qztpg"]
Sep 10 17:43:19.006826 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:19.006806    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2619ad-434a-4de9-924d-807a32e8541d" path="/var/lib/kubelet/pods/9e2619ad-434a-4de9-924d-807a32e8541d/volumes"
Sep 10 17:43:26.895806 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:26.895776    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"
Sep 10 17:43:29.174046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:29.174016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:29.269438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:29.269407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:29.691218 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:29.691184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:42.192690 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.192655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:42.334430 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.334402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:42.427106 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.427073    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-7kpwd"]
Sep 10 17:43:42.427372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.427360    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2619ad-434a-4de9-924d-807a32e8541d" containerName="authorino"
Sep 10 17:43:42.427372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.427373    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2619ad-434a-4de9-924d-807a32e8541d" containerName="authorino"
Sep 10 17:43:42.427482 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.427426    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2619ad-434a-4de9-924d-807a32e8541d" containerName="authorino"
Sep 10 17:43:42.431678 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.431664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:42.434844 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.434823    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dhx2d\""
Sep 10 17:43:42.437828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.437806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-7kpwd"]
Sep 10 17:43:42.450892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.450833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf955\" (UniqueName: \"kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955\") pod \"authorino-557f876f69-7kpwd\" (UID: \"ae1b6267-c441-438f-9145-c1c826a67ae6\") " pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:42.543703 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.543680    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-7kpwd"]
Sep 10 17:43:42.543880 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:43:42.543864    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rf955], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-7kpwd" podUID="ae1b6267-c441-438f-9145-c1c826a67ae6"
Sep 10 17:43:42.551686 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.551668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf955\" (UniqueName: \"kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955\") pod \"authorino-557f876f69-7kpwd\" (UID: \"ae1b6267-c441-438f-9145-c1c826a67ae6\") " pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:42.564164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:42.564133    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf955\" (UniqueName: \"kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955\") pod \"authorino-557f876f69-7kpwd\" (UID: \"ae1b6267-c441-438f-9145-c1c826a67ae6\") " pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:43.001358 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:43.001325    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:43.009092 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:43.009072    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:43.055256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:43.055227    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf955\" (UniqueName: \"kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955\") pod \"ae1b6267-c441-438f-9145-c1c826a67ae6\" (UID: \"ae1b6267-c441-438f-9145-c1c826a67ae6\") "
Sep 10 17:43:43.057286 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:43.057262    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955" (OuterVolumeSpecName: "kube-api-access-rf955") pod "ae1b6267-c441-438f-9145-c1c826a67ae6" (UID: "ae1b6267-c441-438f-9145-c1c826a67ae6"). InnerVolumeSpecName "kube-api-access-rf955". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:43:43.156004 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:43.155975    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf955\" (UniqueName: \"kubernetes.io/projected/ae1b6267-c441-438f-9145-c1c826a67ae6-kube-api-access-rf955\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:43:44.005318 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:44.005293    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-7kpwd"
Sep 10 17:43:44.038068 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:44.038045    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-7kpwd"]
Sep 10 17:43:44.040632 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:44.040592    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-7kpwd"]
Sep 10 17:43:45.006357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:45.006323    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1b6267-c441-438f-9145-c1c826a67ae6" path="/var/lib/kubelet/pods/ae1b6267-c441-438f-9145-c1c826a67ae6/volumes"
Sep 10 17:43:50.036621 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:50.036537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.148438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:50.148409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.232653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:50.232616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.334417 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:50.334347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:50.592174 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:50.592101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:53.875044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:53.875003    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:43:53.989669 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:43:53.989639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:25.728064 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:25.728034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:25.890782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:25.890749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:27.477828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:27.477776    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:27.654487 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:27.654416    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:27.773036 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:27.772939    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:27.906520 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:27.906494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:28.218145 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:28.218103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:28.347683 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:28.347627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:28.551891 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:28.551809    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:44:39.295419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:39.295390    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:39.428693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:39.428661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:58.305201 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:58.305157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:44:58.731943 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:44:58.731910    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:45:12.650694 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:12.650658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:13.256310 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:13.256272    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:45:20.634843 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.634768    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"]
Sep 10 17:45:20.637261 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.637237    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.640098 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.640080    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 17:45:20.640419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.640395    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 17:45:20.640747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.640731    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 17:45:20.641360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.641339    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pntqt\""
Sep 10 17:45:20.651301 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.651281    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"]
Sep 10 17:45:20.689807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.689784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:20.755370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755350    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1716aa43-1264-49b5-9024-35f0dc96f739-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.755496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.755496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.755496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.755496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc44h\" (UniqueName: \"kubernetes.io/projected/1716aa43-1264-49b5-9024-35f0dc96f739-kube-api-access-qc44h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.755650 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.755518    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856734 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1716aa43-1264-49b5-9024-35f0dc96f739-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856784    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856800    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc44h\" (UniqueName: \"kubernetes.io/projected/1716aa43-1264-49b5-9024-35f0dc96f739-kube-api-access-qc44h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.856838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.856831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.857134 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.857114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.857221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.857203    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.857277 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.857262    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.859179 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.859152    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1716aa43-1264-49b5-9024-35f0dc96f739-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.859309 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.859292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1716aa43-1264-49b5-9024-35f0dc96f739-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.865982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.865963    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc44h\" (UniqueName: \"kubernetes.io/projected/1716aa43-1264-49b5-9024-35f0dc96f739-kube-api-access-qc44h\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m\" (UID: \"1716aa43-1264-49b5-9024-35f0dc96f739\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:20.948046 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:20.947997    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:21.237277 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:21.237253    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"]
Sep 10 17:45:21.239715 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:45:21.239688    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1716aa43_1264_49b5_9024_35f0dc96f739.slice/crio-f35c52f2c62a4cc8cc6b1d39b61c7c4d0e297814c6e3c6e70f0d5f4252c22ecd WatchSource:0}: Error finding container f35c52f2c62a4cc8cc6b1d39b61c7c4d0e297814c6e3c6e70f0d5f4252c22ecd: Status 404 returned error can't find the container with id f35c52f2c62a4cc8cc6b1d39b61c7c4d0e297814c6e3c6e70f0d5f4252c22ecd
Sep 10 17:45:21.389621 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:21.389589    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m" event={"ID":"1716aa43-1264-49b5-9024-35f0dc96f739","Type":"ContainerStarted","Data":"f35c52f2c62a4cc8cc6b1d39b61c7c4d0e297814c6e3c6e70f0d5f4252c22ecd"}
Sep 10 17:45:21.622514 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:21.622481    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:45:23.400069 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:23.400030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m" event={"ID":"1716aa43-1264-49b5-9024-35f0dc96f739","Type":"ContainerStarted","Data":"a9730c5e40c7b71c6e7b7791906fd561abbd17384b25742b5e09c52b3eb6df84"}
Sep 10 17:45:23.421243 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:23.421199    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m" podStartSLOduration=2.276698626 podStartE2EDuration="3.421184323s" totalImagesPullingTime="1.144485697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:45:23.420076377 +0000 UTC m=+538.937756540" watchObservedRunningTime="2026-09-10 17:45:23.421184323 +0000 UTC m=+538.938864484"
Sep 10 17:45:26.766780 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:26.766748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:28.219245 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:28.219211    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:45:30.948937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:30.948902    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:30.949446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:30.948966    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:30.961433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:30.961412    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:31.442808 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:31.442780    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m"
Sep 10 17:45:47.858142 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:47.858097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:45:49.335840 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:49.335810    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:45:56.562970 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:45:56.562937    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:46:06.028717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.028681    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"]
Sep 10 17:46:06.032075 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.032058    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.035400 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.035382    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 17:46:06.046181 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.046160    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"]
Sep 10 17:46:06.062622 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.062604    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:46:06.075949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.075927    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.076038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.075956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gth7c\" (UniqueName: \"kubernetes.io/projected/9c96a912-111d-4577-8119-73003a210757-kube-api-access-gth7c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.076038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.075980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.076118 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.076039    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c96a912-111d-4577-8119-73003a210757-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.076118 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.076069    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.076118 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.076085    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177289 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177263    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gth7c\" (UniqueName: \"kubernetes.io/projected/9c96a912-111d-4577-8119-73003a210757-kube-api-access-gth7c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177320    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177359    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c96a912-111d-4577-8119-73003a210757-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177379 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177379    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177393    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177677    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177751    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.177817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.177796    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.179662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.179638    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c96a912-111d-4577-8119-73003a210757-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.179892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.179876    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c96a912-111d-4577-8119-73003a210757-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.187807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.187788    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gth7c\" (UniqueName: \"kubernetes.io/projected/9c96a912-111d-4577-8119-73003a210757-kube-api-access-gth7c\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-twhhg\" (UID: \"9c96a912-111d-4577-8119-73003a210757\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.342470 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.342403    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:06.615989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:06.615855    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"]
Sep 10 17:46:06.618107 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:46:06.618078    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c96a912_111d_4577_8119_73003a210757.slice/crio-7f2215fd010a57015e5ff88487abb7a8a4c4ddaef6a7e7cee536582e742f8252 WatchSource:0}: Error finding container 7f2215fd010a57015e5ff88487abb7a8a4c4ddaef6a7e7cee536582e742f8252: Status 404 returned error can't find the container with id 7f2215fd010a57015e5ff88487abb7a8a4c4ddaef6a7e7cee536582e742f8252
Sep 10 17:46:07.584987 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:07.584950    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg" event={"ID":"9c96a912-111d-4577-8119-73003a210757","Type":"ContainerStarted","Data":"e1cc30726f4f107d5801df166101f649f3130d7c58d30ed7d23fa7e46569b765"}
Sep 10 17:46:07.585336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:07.585009    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg" event={"ID":"9c96a912-111d-4577-8119-73003a210757","Type":"ContainerStarted","Data":"7f2215fd010a57015e5ff88487abb7a8a4c4ddaef6a7e7cee536582e742f8252"}
Sep 10 17:46:07.602511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:07.602464    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg" podStartSLOduration=1.398511393 podStartE2EDuration="1.602436703s" totalImagesPullingTime="203.92531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:07.60053936 +0000 UTC m=+583.118219526" watchObservedRunningTime="2026-09-10 17:46:07.602436703 +0000 UTC m=+583.120116864"
Sep 10 17:46:07.927604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:07.927573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:46:15.813282 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.813248    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"]
Sep 10 17:46:15.816516 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.816502    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.819002 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.818977    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 17:46:15.828289 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.828264    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"]
Sep 10 17:46:15.856094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.856074    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:46:15.950267 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950246    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.950374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.950374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950311    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.950519 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.950519 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:15.950612 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:15.950532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggxxp\" (UniqueName: \"kubernetes.io/projected/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-kube-api-access-ggxxp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.051244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.051223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.051365 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.051303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.051699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.051672    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.051857 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.051709    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.051962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.051845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.052168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.052140    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.052369 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.052347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggxxp\" (UniqueName: \"kubernetes.io/projected/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-kube-api-access-ggxxp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.052489 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.052406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.052772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.052752    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.054127 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.054104    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.054127 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.054110    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.062541 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.062518    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggxxp\" (UniqueName: \"kubernetes.io/projected/7772eb5a-b6b8-499c-b70f-5856d1a8cb8f-kube-api-access-ggxxp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jlwql\" (UID: \"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.126099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.126078    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:16.343508 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.343472    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:16.343655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.343529    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:16.355842 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.355821    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:16.398398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.398372    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"]
Sep 10 17:46:16.400858 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:46:16.400835    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7772eb5a_b6b8_499c_b70f_5856d1a8cb8f.slice/crio-ed156f00c0b7928263809bf1be4fb4b558f4f0e10e650e4f2af9c2911397b57d WatchSource:0}: Error finding container ed156f00c0b7928263809bf1be4fb4b558f4f0e10e650e4f2af9c2911397b57d: Status 404 returned error can't find the container with id ed156f00c0b7928263809bf1be4fb4b558f4f0e10e650e4f2af9c2911397b57d
Sep 10 17:46:16.623826 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.623800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql" event={"ID":"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f","Type":"ContainerStarted","Data":"ed156f00c0b7928263809bf1be4fb4b558f4f0e10e650e4f2af9c2911397b57d"}
Sep 10 17:46:16.636139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:16.636120    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-twhhg"
Sep 10 17:46:17.629534 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:17.629493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql" event={"ID":"7772eb5a-b6b8-499c-b70f-5856d1a8cb8f","Type":"ContainerStarted","Data":"ccf831b844f9afd9a6e72259f491006744bb6b64ea7912088662db26645db93b"}
Sep 10 17:46:17.661911 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:17.661856    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql" podStartSLOduration=2.479049783 podStartE2EDuration="2.661841781s" totalImagesPullingTime="182.791998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:46:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:46:17.660651773 +0000 UTC m=+593.178331934" watchObservedRunningTime="2026-09-10 17:46:17.661841781 +0000 UTC m=+593.179521937"
Sep 10 17:46:17.941716 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:17.941649    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:46:26.126668 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:26.126630    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:26.127121 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:26.126686    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:26.139136 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:26.139115    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:26.677715 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:26.677690    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jlwql"
Sep 10 17:46:50.301109 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:46:50.301024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:12.159522 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:12.159480    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:35.824725 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.824692    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-856bfbb5f6-r8jtd"]
Sep 10 17:47:35.827990 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.827970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:35.831594 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.831572    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dhx2d\""
Sep 10 17:47:35.831696 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.831574    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 17:47:35.843493 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.843472    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-856bfbb5f6-r8jtd"]
Sep 10 17:47:35.959035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.959004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln6tz\" (UniqueName: \"kubernetes.io/projected/147919b5-2162-4add-9224-eb9f5f0ddf3c-kube-api-access-ln6tz\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:35.959195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:35.959112    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/147919b5-2162-4add-9224-eb9f5f0ddf3c-tls-cert\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.060282 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.060256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln6tz\" (UniqueName: \"kubernetes.io/projected/147919b5-2162-4add-9224-eb9f5f0ddf3c-kube-api-access-ln6tz\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.060407 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.060313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/147919b5-2162-4add-9224-eb9f5f0ddf3c-tls-cert\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.062841 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.062814    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/147919b5-2162-4add-9224-eb9f5f0ddf3c-tls-cert\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.070315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.070292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln6tz\" (UniqueName: \"kubernetes.io/projected/147919b5-2162-4add-9224-eb9f5f0ddf3c-kube-api-access-ln6tz\") pod \"authorino-856bfbb5f6-r8jtd\" (UID: \"147919b5-2162-4add-9224-eb9f5f0ddf3c\") " pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.136397 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.136377    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-856bfbb5f6-r8jtd"
Sep 10 17:47:36.422060 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.421995    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-856bfbb5f6-r8jtd"]
Sep 10 17:47:36.425770 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:47:36.425747    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod147919b5_2162_4add_9224_eb9f5f0ddf3c.slice/crio-639b8d44e6ab129f09f21e8116f1ad9fa934c44752d18dd13a34493cd93154f9 WatchSource:0}: Error finding container 639b8d44e6ab129f09f21e8116f1ad9fa934c44752d18dd13a34493cd93154f9: Status 404 returned error can't find the container with id 639b8d44e6ab129f09f21e8116f1ad9fa934c44752d18dd13a34493cd93154f9
Sep 10 17:47:36.427443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.427428    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:47:36.964012 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:36.963976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-856bfbb5f6-r8jtd" event={"ID":"147919b5-2162-4add-9224-eb9f5f0ddf3c","Type":"ContainerStarted","Data":"639b8d44e6ab129f09f21e8116f1ad9fa934c44752d18dd13a34493cd93154f9"}
Sep 10 17:47:37.969944 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:37.969902    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-856bfbb5f6-r8jtd" event={"ID":"147919b5-2162-4add-9224-eb9f5f0ddf3c","Type":"ContainerStarted","Data":"b12f1dcfb52d460b15c77f5f757597fd980bc798ff60a259859c03d64c5b3a3b"}
Sep 10 17:47:37.999827 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:37.999768    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-856bfbb5f6-r8jtd" podStartSLOduration=2.206266442 podStartE2EDuration="2.999745973s" totalImagesPullingTime="793.479531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:47:37.99872754 +0000 UTC m=+673.516407704" watchObservedRunningTime="2026-09-10 17:47:37.999745973 +0000 UTC m=+673.517426137"
Sep 10 17:47:38.591853 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:38.591799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:38.691259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:38.691225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:39.874574 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:39.874542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:40.049446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:40.049414    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:47:49.193614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:47:49.193576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:00.102039 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:00.102007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:04.083872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:04.083842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:04.187811 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:04.187774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:04.447855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:04.447823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:04.657800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:04.657760    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:05.595894 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:05.595858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:05.712111 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:05.712060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:06.181061 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:06.181019    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:07.702052 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:07.702017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:07.801686 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:07.801655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:08.043855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:08.043780    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:48:08.866554 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:08.866518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:08.933056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:08.933008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:09.337090 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:09.337004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:09.488259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:09.488227    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:09.588829 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:09.588743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:12.344880 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:12.344848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:12.532763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:12.532731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:12.635007 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:12.634979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:12.731902 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:12.731874    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:12.831900 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:12.831875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:13.138041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:13.138004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:13.243688 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:13.243652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:13.892348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:13.892306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:15.689210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:15.689160    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:15.795780 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:15.795750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:16.861339 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:16.861303    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:16.936633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:16.936596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:17.040491 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:17.040446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:17.442157 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:17.442127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:17.535767 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:17.535735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:18.650089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:18.650058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:18.765340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:18.765294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:20.288750 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:20.288723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:20.391881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:20.391847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:20.495782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:20.495751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:20.848356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:20.848321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:20.935470 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:20.935409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:21.639741 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:21.639708    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:48:23.973828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:23.973799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:24.141738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:24.141705    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:24.248248 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:24.248176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:24.347526 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:24.347497    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:27.119999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:27.119965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:27.526964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:27.526880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:27.659640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:27.659609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:27.782317 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:27.782242    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:28.836847 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:28.836760    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:30.287520 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:30.287489    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:32.660292 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:32.660259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:33.032777 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:33.032700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:33.477074 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:33.477042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:33.773141 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:33.773043    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:35.012246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:35.012211    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:35.134260 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:35.134230    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:35.234762 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:35.234730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:35.422598 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:35.422566    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:36.975579 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:36.975540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:37.050000 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:37.049967    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:48:37.071507 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:37.071475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:38.174401 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.174374    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:48:38.177726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.177707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.180747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.180727    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:48:38.181560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.181541    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tdclw\""
Sep 10 17:48:38.181659 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.181595    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:48:38.181659 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.181602    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-783a5e-serving-cert\""
Sep 10 17:48:38.181887 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.181871    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:48:38.193292 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.193266    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:48:38.221505 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.221478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:38.271092 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.271071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.271189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.271105    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.271189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.271164    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldng\" (UniqueName: \"kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.271291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.271194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.301175 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.301153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:38.372355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.372333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldng\" (UniqueName: \"kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.372446 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.372361    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.372518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.372500    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.372566 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.372543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.372679 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:48:38.372663    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-serving-cert: secret "maas-api-e2e-inf-b-783a5e-serving-cert" not found
Sep 10 17:48:38.372762 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:48:38.372750    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls podName:e21f10f1-1384-4c74-9007-9d409f49b2b7 nodeName:}" failed. No retries permitted until 2026-09-10 17:48:38.87272833 +0000 UTC m=+734.390408476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls") pod "maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7") : secret "maas-api-e2e-inf-b-783a5e-serving-cert" not found
Sep 10 17:48:38.373164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.373147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.374992 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.374971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.381092 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.381063    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldng\" (UniqueName: \"kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.877392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.877360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:38.877577 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:48:38.877524    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-serving-cert: secret "maas-api-e2e-inf-b-783a5e-serving-cert" not found
Sep 10 17:48:38.877627 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:48:38.877592    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls podName:e21f10f1-1384-4c74-9007-9d409f49b2b7 nodeName:}" failed. No retries permitted until 2026-09-10 17:48:39.877576033 +0000 UTC m=+735.395256173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls") pod "maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7") : secret "maas-api-e2e-inf-b-783a5e-serving-cert" not found
Sep 10 17:48:38.897256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.897226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:38.996120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:38.996088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:39.287246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:39.287162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:39.886100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:39.886066    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:39.888667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:39.888645    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:39.987330 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:39.987297    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:40.102228 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:40.102194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:40.287267 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:40.287233    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:48:40.290004 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:48:40.289973    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21f10f1_1384_4c74_9007_9d409f49b2b7.slice/crio-2cdcfc49c7caf51bfdbec46d48327b533c5bc6ebb4cb2bf1d8713b27405b6742 WatchSource:0}: Error finding container 2cdcfc49c7caf51bfdbec46d48327b533c5bc6ebb4cb2bf1d8713b27405b6742: Status 404 returned error can't find the container with id 2cdcfc49c7caf51bfdbec46d48327b533c5bc6ebb4cb2bf1d8713b27405b6742
Sep 10 17:48:41.237792 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:41.237757    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" event={"ID":"e21f10f1-1384-4c74-9007-9d409f49b2b7","Type":"ContainerStarted","Data":"2cdcfc49c7caf51bfdbec46d48327b533c5bc6ebb4cb2bf1d8713b27405b6742"}
Sep 10 17:48:41.346336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:41.346293    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:42.044180 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:42.044141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:42.339860 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:42.339771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:42.430704 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:42.430663    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:43.088947 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:43.088913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:43.253748 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:43.253722    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" event={"ID":"e21f10f1-1384-4c74-9007-9d409f49b2b7","Type":"ContainerStarted","Data":"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"}
Sep 10 17:48:43.253912 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:43.253771    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:43.276145 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:43.276102    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" podStartSLOduration=3.066468929 podStartE2EDuration="5.276087972s" totalImagesPullingTime="2.209619043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:48:43.273699561 +0000 UTC m=+738.791379723" watchObservedRunningTime="2026-09-10 17:48:43.276087972 +0000 UTC m=+738.793768134"
Sep 10 17:48:45.766049 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:45.766018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:48:49.263028 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:49.263000    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:48:53.840049 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:53.840014    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:48:59.374803 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:48:59.374766    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:00.321804 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:00.321768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:00.422081 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:00.422040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:00.522141 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:00.522113    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:00.639253 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:00.639225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:00.867495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:00.867443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:01.024298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:01.024214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:01.434366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:01.434334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:01.719765 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:01.719652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:01.817739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:01.817713    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:01.986195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:01.986075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:02.079657 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:02.079634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:02.348826 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:02.348757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:04.952853 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.952818    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:49:04.959609 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.959589    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.968168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.968149    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-7e6ec3a2-kserve-self-signed-certs\""
Sep 10 17:49:04.970027 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.970005    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:49:04.980846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.980825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.980937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.980860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9w8j\" (UniqueName: \"kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.980937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.980883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.980937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.980910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.981080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.980950    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:04.981080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:04.981007    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.017342 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.017318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.082015 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.081991    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082101 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082025    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082167 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082209 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082196    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9w8j\" (UniqueName: \"kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082334 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082318    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082383 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082426 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082405    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.082495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.082443    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.084669 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.084648    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.084891 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.084875    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.099193 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.099163    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9w8j\" (UniqueName: \"kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j\") pod \"e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") " pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.131942 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.131915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.229610 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.229541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.237545 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.237520    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:05.240970 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.240952    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.243719 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.243695    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-624b5c35-serving-cert\""
Sep 10 17:49:05.251782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.251764    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:05.268660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.268640    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:05.283854 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.283833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.283960 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.283881    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.283960 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.283919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb2bj\" (UniqueName: \"kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.284056 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.283987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.314153 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.311669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.385158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.385127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.385328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.385209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.385328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.385280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb2bj\" (UniqueName: \"kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.385328 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.385311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.385547 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:49:05.385354    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-serving-cert: secret "maas-api-e2e-migrate-624b5c35-serving-cert" not found
Sep 10 17:49:05.385547 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:49:05.385411    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls podName:39156512-f5c7-478b-a167-c051579f4160 nodeName:}" failed. No retries permitted until 2026-09-10 17:49:05.885395486 +0000 UTC m=+761.403075629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls") pod "maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" (UID: "39156512-f5c7-478b-a167-c051579f4160") : secret "maas-api-e2e-migrate-624b5c35-serving-cert" not found
Sep 10 17:49:05.385793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.385774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.387745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.387728    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.394755 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.394737    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb2bj\" (UniqueName: \"kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.494410 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.494346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.546912 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.546890    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:49:05.549613 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:49:05.549590    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2871a189_2d25_484e_8c12_0f4ff2ba7f95.slice/crio-9d882e52f978ffc1ae0b32891b796c507630c3d951a1cfcb6e2998f11b16dc9a WatchSource:0}: Error finding container 9d882e52f978ffc1ae0b32891b796c507630c3d951a1cfcb6e2998f11b16dc9a: Status 404 returned error can't find the container with id 9d882e52f978ffc1ae0b32891b796c507630c3d951a1cfcb6e2998f11b16dc9a
Sep 10 17:49:05.883016 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.882986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:05.891074 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.891055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.893828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.893804    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") pod \"maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:05.983656 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:05.983626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:06.153313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:06.153244    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:06.360196 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:06.360160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" event={"ID":"2871a189-2d25-484e-8c12-0f4ff2ba7f95","Type":"ContainerStarted","Data":"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"}
Sep 10 17:49:06.360372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:06.360207    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" event={"ID":"2871a189-2d25-484e-8c12-0f4ff2ba7f95","Type":"ContainerStarted","Data":"9d882e52f978ffc1ae0b32891b796c507630c3d951a1cfcb6e2998f11b16dc9a"}
Sep 10 17:49:06.381583 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:06.381519    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" podStartSLOduration=2.381501513 podStartE2EDuration="2.381501513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:06.379893992 +0000 UTC m=+761.897574155" watchObservedRunningTime="2026-09-10 17:49:06.381501513 +0000 UTC m=+761.899181673"
Sep 10 17:49:06.428644 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:06.428518    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:06.430812 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:49:06.430785    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39156512_f5c7_478b_a167_c051579f4160.slice/crio-1f42aa1ee3a7c65a74a953c874398588b4110132bda76da534de443dea01b22a WatchSource:0}: Error finding container 1f42aa1ee3a7c65a74a953c874398588b4110132bda76da534de443dea01b22a: Status 404 returned error can't find the container with id 1f42aa1ee3a7c65a74a953c874398588b4110132bda76da534de443dea01b22a
Sep 10 17:49:07.366284 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:07.366252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" event={"ID":"39156512-f5c7-478b-a167-c051579f4160","Type":"ContainerStarted","Data":"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"}
Sep 10 17:49:07.366744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:07.366301    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" event={"ID":"39156512-f5c7-478b-a167-c051579f4160","Type":"ContainerStarted","Data":"1f42aa1ee3a7c65a74a953c874398588b4110132bda76da534de443dea01b22a"}
Sep 10 17:49:07.393268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:07.393225    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" podStartSLOduration=1.871526799 podStartE2EDuration="2.393210735s" totalImagesPullingTime="521.683936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:07.390922548 +0000 UTC m=+762.908602709" watchObservedRunningTime="2026-09-10 17:49:07.393210735 +0000 UTC m=+762.910890896"
Sep 10 17:49:08.032185 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:08.032152    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:49:08.371089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:08.371047    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:14.380398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:14.380366    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:15.269148 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:15.269113    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:15.269350 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:15.269162    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:15.281855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:15.281828    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:15.411217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:15.411190    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:49:22.412154 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.412122    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:49:22.415259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.415245    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.418124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.418100    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-51ed8d\"/\"openshift-service-ca.crt\""
Sep 10 17:49:22.419162 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.419136    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-51ed8d\"/\"default-dockercfg-gskts\""
Sep 10 17:49:22.419256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.419157    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-51ed8d\"/\"kube-root-ca.crt\""
Sep 10 17:49:22.419256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.419212    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-51ed8d\"/\"autores-model-51ed8d-kserve-self-signed-certs\""
Sep 10 17:49:22.428595 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.428572    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:49:22.522325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:22.522508 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.522575 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522516    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.522575 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.522675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522616    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9cl\" (UniqueName: \"kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.522675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.522774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.522756    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623625 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623603    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623673    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623699    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623745 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9cl\" (UniqueName: \"kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.623940 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.623797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.624049 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.624028    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.624110 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.624088    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.624200 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.624182    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.626109 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.626087    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.626226 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.626212    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.633583 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.633561    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9cl\" (UniqueName: \"kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl\") pod \"autores-model-51ed8d-kserve-b6649b78c-cs864\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") " pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:22.724425 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:22.724369    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:23.007527 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:49:23.007494    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e5360e7_8b15_4696_b9e6_5d96b2f338d4.slice/crio-c73ace7f982fc630fa5e65f4a0837d252b8c5b9fefcc7d5a839590e69fbbbf58 WatchSource:0}: Error finding container c73ace7f982fc630fa5e65f4a0837d252b8c5b9fefcc7d5a839590e69fbbbf58: Status 404 returned error can't find the container with id c73ace7f982fc630fa5e65f4a0837d252b8c5b9fefcc7d5a839590e69fbbbf58
Sep 10 17:49:23.009211 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:23.009067    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:49:23.438983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:23.438943    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" event={"ID":"3e5360e7-8b15-4696-b9e6-5d96b2f338d4","Type":"ContainerStarted","Data":"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"}
Sep 10 17:49:23.438983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:23.438987    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" event={"ID":"3e5360e7-8b15-4696-b9e6-5d96b2f338d4","Type":"ContainerStarted","Data":"c73ace7f982fc630fa5e65f4a0837d252b8c5b9fefcc7d5a839590e69fbbbf58"}
Sep 10 17:49:23.475272 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:23.475222    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" podStartSLOduration=1.4752075900000001 podStartE2EDuration="1.47520759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:23.473091569 +0000 UTC m=+778.990771730" watchObservedRunningTime="2026-09-10 17:49:23.47520759 +0000 UTC m=+778.992887809"
Sep 10 17:49:26.456102 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:26.456069    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:49:32.237634 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.237600    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:49:32.241037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.241017    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.244831 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.244801    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48d00b\"/\"kube-root-ca.crt\""
Sep 10 17:49:32.245872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.245848    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48d00b\"/\"test-model-48d00b-kserve-self-signed-certs\""
Sep 10 17:49:32.245975 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.245886    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-48d00b\"/\"default-dockercfg-s5zlf\""
Sep 10 17:49:32.245975 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.245896    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-48d00b\"/\"openshift-service-ca.crt\""
Sep 10 17:49:32.261517 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.261495    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:49:32.293499 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.293470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:32.305838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.305811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.305946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.305853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz48w\" (UniqueName: \"kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.305946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.305896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.305946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.305931    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.306158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.305964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.306158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.306074    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407466 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407424    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz48w\" (UniqueName: \"kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407595    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407911 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407891    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407961 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407931    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.407999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.407986    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.409871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.409852    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.410139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.410122    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.416263 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.416241    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz48w\" (UniqueName: \"kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w\") pod \"test-model-48d00b-kserve-7d68b97b9c-jp6jz\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") " pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.550499 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.550405    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:32.725042 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.725009    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:32.725211 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.725060    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:32.737170 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.737148    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:32.827012 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:32.826948    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:49:32.829878 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:49:32.829851    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfef4c1cb_522d_47b3_aab5_1c37cb28b0bb.slice/crio-166d2057ff22a1270c5c44cb56e453d57dbbb0973e613c691bb12679b16bd5ed WatchSource:0}: Error finding container 166d2057ff22a1270c5c44cb56e453d57dbbb0973e613c691bb12679b16bd5ed: Status 404 returned error can't find the container with id 166d2057ff22a1270c5c44cb56e453d57dbbb0973e613c691bb12679b16bd5ed
Sep 10 17:49:33.487655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:33.487624    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" event={"ID":"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb","Type":"ContainerStarted","Data":"2b430c36472d5e6576115b3a003c0f56b3aa5fad229afbfd4f2c6be122f268c2"}
Sep 10 17:49:33.487655 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:33.487658    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" event={"ID":"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb","Type":"ContainerStarted","Data":"166d2057ff22a1270c5c44cb56e453d57dbbb0973e613c691bb12679b16bd5ed"}
Sep 10 17:49:33.499496 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:33.499475    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:49:33.506243 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:33.506205    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" podStartSLOduration=1.506192349 podStartE2EDuration="1.506192349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:49:33.50473014 +0000 UTC m=+789.022410303" watchObservedRunningTime="2026-09-10 17:49:33.506192349 +0000 UTC m=+789.023872510"
Sep 10 17:49:36.880320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:36.880274    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:36.880972 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:36.880506    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" podUID="39156512-f5c7-478b-a167-c051579f4160" containerName="maas-api" containerID="cri-o://93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2" gracePeriod=30
Sep 10 17:49:36.938817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:36.938790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:37.059156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.059116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:37.122538 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.122517    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:37.158275 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.158212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:37.237902 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.237861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:37.249048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.249027    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb2bj\" (UniqueName: \"kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj\") pod \"39156512-f5c7-478b-a167-c051579f4160\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") "
Sep 10 17:49:37.249122 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.249070    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca\") pod \"39156512-f5c7-478b-a167-c051579f4160\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") "
Sep 10 17:49:37.249122 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.249109    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs\") pod \"39156512-f5c7-478b-a167-c051579f4160\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") "
Sep 10 17:49:37.249209 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.249133    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") pod \"39156512-f5c7-478b-a167-c051579f4160\" (UID: \"39156512-f5c7-478b-a167-c051579f4160\") "
Sep 10 17:49:37.249404 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.249382    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39156512-f5c7-478b-a167-c051579f4160" (UID: "39156512-f5c7-478b-a167-c051579f4160"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:49:37.251282 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.251254    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "39156512-f5c7-478b-a167-c051579f4160" (UID: "39156512-f5c7-478b-a167-c051579f4160"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:49:37.251371 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.251289    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39156512-f5c7-478b-a167-c051579f4160" (UID: "39156512-f5c7-478b-a167-c051579f4160"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:49:37.251371 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.251314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj" (OuterVolumeSpecName: "kube-api-access-jb2bj") pod "39156512-f5c7-478b-a167-c051579f4160" (UID: "39156512-f5c7-478b-a167-c051579f4160"). InnerVolumeSpecName "kube-api-access-jb2bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:49:37.350520 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.350495    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39156512-f5c7-478b-a167-c051579f4160-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:49:37.350520 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.350518    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:49:37.350684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.350528    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39156512-f5c7-478b-a167-c051579f4160-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:49:37.350684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.350535    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb2bj\" (UniqueName: \"kubernetes.io/projected/39156512-f5c7-478b-a167-c051579f4160-kube-api-access-jb2bj\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:49:37.393863 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.393822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:37.508156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.508082    2571 generic.go:356] "Generic (PLEG): container finished" podID="39156512-f5c7-478b-a167-c051579f4160" containerID="93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2" exitCode=0
Sep 10 17:49:37.508156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.508127    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"
Sep 10 17:49:37.508357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.508155    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" event={"ID":"39156512-f5c7-478b-a167-c051579f4160","Type":"ContainerDied","Data":"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"}
Sep 10 17:49:37.508357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.508206    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77" event={"ID":"39156512-f5c7-478b-a167-c051579f4160","Type":"ContainerDied","Data":"1f42aa1ee3a7c65a74a953c874398588b4110132bda76da534de443dea01b22a"}
Sep 10 17:49:37.508357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.508237    2571 scope.go:117] "RemoveContainer" containerID="93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"
Sep 10 17:49:37.517972 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.517952    2571 scope.go:117] "RemoveContainer" containerID="93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"
Sep 10 17:49:37.518241 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:49:37.518224    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2\": container with ID starting with 93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2 not found: ID does not exist" containerID="93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"
Sep 10 17:49:37.518291 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.518250    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2"} err="failed to get container status \"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2\": rpc error: code = NotFound desc = could not find container \"93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2\": container with ID starting with 93ff4164e4e1551d523e0d98b8b7bc1d370a785d9fa35980c9fa165a64c018b2 not found: ID does not exist"
Sep 10 17:49:37.529771 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.529752    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:37.538467 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.538436    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-624b5c35-f68799d6d-5rg77"]
Sep 10 17:49:37.856538 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:37.856447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:38.181437 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:38.181393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:38.261964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:38.261934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:39.007660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:39.007625    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39156512-f5c7-478b-a167-c051579f4160" path="/var/lib/kubelet/pods/39156512-f5c7-478b-a167-c051579f4160/volumes"
Sep 10 17:49:41.064505 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:41.064472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:41.177348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:41.177313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:42.310250 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.310215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:42.451725 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.451691    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:42.550628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.550593    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:42.550628 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.550633    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:42.563283 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.563218    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:42.852948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.852874    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:42.950058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:42.950030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:43.551607 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:43.551572    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:49:47.513651 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:47.513566    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:49:49.526836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:49.526799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:49:53.996667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:49:53.996621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:04.570416 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:04.570384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:05.489549 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:05.489518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:05.597930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:05.597899    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:05.694223 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:05.694194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:05.793207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:05.793137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:05.870472 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:05.870426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:06.219645 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:06.219612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:09.334986 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:09.334947    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:50:11.192761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:11.192724    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:50:11.248963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:11.248932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:12.953359 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:12.953321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:13.056469 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:13.056408    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:13.177172 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:13.177139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:13.470139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:13.470107    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:14.560777 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:14.560741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:15.027844 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:15.027811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:15.349396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:15.349323    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:15.612700 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:15.612666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:18.250245 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:18.250217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:18.343024 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:18.342990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:19.610340 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:19.610309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:19.713160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:19.713135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:19.866009 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:19.865986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:20.211088 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:20.211008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:20.314130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:20.314101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:26.212977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.212927    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" podUID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" containerName="main" containerID="cri-o://eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec" gracePeriod=45
Sep 10 17:50:26.466976 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.466924    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:50:26.554016 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.553987    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554046    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554080    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554108    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554164 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554142    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554168    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g9cl\" (UniqueName: \"kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl\") pod \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\" (UID: \"3e5360e7-8b15-4696-b9e6-5d96b2f338d4\") "
Sep 10 17:50:26.554412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554204    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:26.554412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554327    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:26.554412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554344    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home" (OuterVolumeSpecName: "home") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:26.554611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554478    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.554611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554493    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.554611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.554507    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.556362 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.556334    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm" (OuterVolumeSpecName: "dshm") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:26.556362 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.556353    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:50:26.556532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.556416    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl" (OuterVolumeSpecName: "kube-api-access-6g9cl") pod "3e5360e7-8b15-4696-b9e6-5d96b2f338d4" (UID: "3e5360e7-8b15-4696-b9e6-5d96b2f338d4"). InnerVolumeSpecName "kube-api-access-6g9cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:26.655634 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.655615    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.655634 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.655634    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g9cl\" (UniqueName: \"kubernetes.io/projected/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-kube-api-access-6g9cl\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.655772 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.655644    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e5360e7-8b15-4696-b9e6-5d96b2f338d4-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:26.741057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.740989    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" containerID="eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec" exitCode=0
Sep 10 17:50:26.741057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.741040    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"
Sep 10 17:50:26.741057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.741046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" event={"ID":"3e5360e7-8b15-4696-b9e6-5d96b2f338d4","Type":"ContainerDied","Data":"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"}
Sep 10 17:50:26.741257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.741082    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864" event={"ID":"3e5360e7-8b15-4696-b9e6-5d96b2f338d4","Type":"ContainerDied","Data":"c73ace7f982fc630fa5e65f4a0837d252b8c5b9fefcc7d5a839590e69fbbbf58"}
Sep 10 17:50:26.741257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.741106    2571 scope.go:117] "RemoveContainer" containerID="eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"
Sep 10 17:50:26.749410 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.749380    2571 scope.go:117] "RemoveContainer" containerID="eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"
Sep 10 17:50:26.749832 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:50:26.749801    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec\": container with ID starting with eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec not found: ID does not exist" containerID="eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"
Sep 10 17:50:26.749931 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.749843    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec"} err="failed to get container status \"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec\": rpc error: code = NotFound desc = could not find container \"eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec\": container with ID starting with eb4563292a00d5e4b984d7aaf2625ec4d2af4975adc75eeb9ac36689da9a76ec not found: ID does not exist"
Sep 10 17:50:26.762706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.762686    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:50:26.765217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:26.765194    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-51ed8d/autores-model-51ed8d-kserve-b6649b78c-cs864"]
Sep 10 17:50:27.009208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:27.009140    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" path="/var/lib/kubelet/pods/3e5360e7-8b15-4696-b9e6-5d96b2f338d4/volumes"
Sep 10 17:50:27.533902 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:27.533867    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:50:30.023199 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023165    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023501    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" containerName="main"
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023513    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" containerName="main"
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023525    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39156512-f5c7-478b-a167-c051579f4160" containerName="maas-api"
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023532    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="39156512-f5c7-478b-a167-c051579f4160" containerName="maas-api"
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023595    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e5360e7-8b15-4696-b9e6-5d96b2f338d4" containerName="main"
Sep 10 17:50:30.023630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.023606    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="39156512-f5c7-478b-a167-c051579f4160" containerName="maas-api"
Sep 10 17:50:30.028011 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.027993    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.030819 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.030794    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-7e6ec3a2-kserve-self-signed-certs\""
Sep 10 17:50:30.043045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.043017    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:50:30.184533 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184505    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.184662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184543    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb8nt\" (UniqueName: \"kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.184662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184569    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.184662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184587    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.184662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184604    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.184873 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.184722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285437    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb8nt\" (UniqueName: \"kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285511    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285564    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285789 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285789 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285789    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.285962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.285905    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.288235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.288205    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.288534 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.288513    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.294567 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.294537    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb8nt\" (UniqueName: \"kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt\") pod \"e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") " pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.336416 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.336390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:30.421927 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.421896    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:30.612776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.612754    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:50:30.615035 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:50:30.615008    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c812c5_ba88_46a3_af00_97cbfb8cb1d6.slice/crio-0b0aa9f1b1852f801ec92a362b394477a6b9949ac4cc6d7f1c8b7020d49fc261 WatchSource:0}: Error finding container 0b0aa9f1b1852f801ec92a362b394477a6b9949ac4cc6d7f1c8b7020d49fc261: Status 404 returned error can't find the container with id 0b0aa9f1b1852f801ec92a362b394477a6b9949ac4cc6d7f1c8b7020d49fc261
Sep 10 17:50:30.763706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.763672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" event={"ID":"35c812c5-ba88-46a3-af00-97cbfb8cb1d6","Type":"ContainerStarted","Data":"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"}
Sep 10 17:50:30.763833 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.763716    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" event={"ID":"35c812c5-ba88-46a3-af00-97cbfb8cb1d6","Type":"ContainerStarted","Data":"0b0aa9f1b1852f801ec92a362b394477a6b9949ac4cc6d7f1c8b7020d49fc261"}
Sep 10 17:50:30.782738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:30.782692    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" podStartSLOduration=0.782675938 podStartE2EDuration="782.675938ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:50:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:50:30.781928106 +0000 UTC m=+846.299608269" watchObservedRunningTime="2026-09-10 17:50:30.782675938 +0000 UTC m=+846.300356100"
Sep 10 17:50:34.602259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:34.602223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:40.337356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:40.337314    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:40.337914 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:40.337379    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:40.350447 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:40.350416    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:40.823239 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:40.823209    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:44.259326 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:44.259292    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:50:44.282556 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:44.282524    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:45.603208 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:45.603177    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:50:45.628600 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:45.628576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:48.159949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:48.159912    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:50:49.022149 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:49.022110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:49.332444 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:49.332367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:49.839617 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:49.839578    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.208846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.208817    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.335220 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.335186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.435728 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.435695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.544259 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.544193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.637429 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.637401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:50.713587 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:50.713558    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:51.512327 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:51.512296    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:51.600881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:51.600855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:51.701759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:51.701735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:53.286828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:53.286792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:53.943402 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:53.943366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:54.040575 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:54.040541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:55.146471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:55.146421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:55.249552 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:55.249519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:55.357442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:55.357414    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:55.648001 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:55.647973    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:55.742026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:55.741998    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:59.240649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.240594    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:59.282999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.282958    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" podUID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" containerName="main" containerID="cri-o://21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba" gracePeriod=45
Sep 10 17:50:59.364189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.364169    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:59.456403 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.456373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:59.526799 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.526781    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:59.560720 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.560693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:50:59.625296 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625276    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625317    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625344    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625384    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb8nt\" (UniqueName: \"kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625617 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625448    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625617 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625502    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs\") pod \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\" (UID: \"35c812c5-ba88-46a3-af00-97cbfb8cb1d6\") "
Sep 10 17:50:59.625617 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625596    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:59.625768 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625621    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home" (OuterVolumeSpecName: "home") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:59.625768 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625748    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:59.625891 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625875    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.625955 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625912    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.625955 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.625926    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.627738 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.627715    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm" (OuterVolumeSpecName: "dshm") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:50:59.627855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.627765    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:50:59.627855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.627822    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt" (OuterVolumeSpecName: "kube-api-access-mb8nt") pod "35c812c5-ba88-46a3-af00-97cbfb8cb1d6" (UID: "35c812c5-ba88-46a3-af00-97cbfb8cb1d6"). InnerVolumeSpecName "kube-api-access-mb8nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:50:59.726277 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.726252    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.726277 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.726276    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb8nt\" (UniqueName: \"kubernetes.io/projected/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-kube-api-access-mb8nt\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.726403 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.726292    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35c812c5-ba88-46a3-af00-97cbfb8cb1d6-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:50:59.902113 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.902086    2571 generic.go:356] "Generic (PLEG): container finished" podID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" containerID="21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba" exitCode=0
Sep 10 17:50:59.902207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.902141    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"
Sep 10 17:50:59.902207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.902174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" event={"ID":"35c812c5-ba88-46a3-af00-97cbfb8cb1d6","Type":"ContainerDied","Data":"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"}
Sep 10 17:50:59.902276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.902218    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7" event={"ID":"35c812c5-ba88-46a3-af00-97cbfb8cb1d6","Type":"ContainerDied","Data":"0b0aa9f1b1852f801ec92a362b394477a6b9949ac4cc6d7f1c8b7020d49fc261"}
Sep 10 17:50:59.902276 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.902236    2571 scope.go:117] "RemoveContainer" containerID="21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"
Sep 10 17:50:59.910937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.910919    2571 scope.go:117] "RemoveContainer" containerID="21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"
Sep 10 17:50:59.911209 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:50:59.911193    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba\": container with ID starting with 21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba not found: ID does not exist" containerID="21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"
Sep 10 17:50:59.911257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.911216    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba"} err="failed to get container status \"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba\": rpc error: code = NotFound desc = could not find container \"21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba\": container with ID starting with 21cd2b63ae4da9e3ffe4232c31b246e47f1ce259937a97148497e69b9f91bdba not found: ID does not exist"
Sep 10 17:50:59.932409 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.932389    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:50:59.946490 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:50:59.946470    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-7e6ec3a2-kserve-746d4d4cc9-56zw7"]
Sep 10 17:51:00.121492 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.121449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:00.623021 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.622978    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" podUID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" containerName="main" containerID="cri-o://b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d" gracePeriod=45
Sep 10 17:51:00.866723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.866702    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:51:00.908220 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.908150    2571 generic.go:356] "Generic (PLEG): container finished" podID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" containerID="b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d" exitCode=0
Sep 10 17:51:00.908220 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.908189    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"
Sep 10 17:51:00.908406 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.908231    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" event={"ID":"2871a189-2d25-484e-8c12-0f4ff2ba7f95","Type":"ContainerDied","Data":"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"}
Sep 10 17:51:00.908406 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.908266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl" event={"ID":"2871a189-2d25-484e-8c12-0f4ff2ba7f95","Type":"ContainerDied","Data":"9d882e52f978ffc1ae0b32891b796c507630c3d951a1cfcb6e2998f11b16dc9a"}
Sep 10 17:51:00.908406 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.908288    2571 scope.go:117] "RemoveContainer" containerID="b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"
Sep 10 17:51:00.916150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.916128    2571 scope.go:117] "RemoveContainer" containerID="b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"
Sep 10 17:51:00.916427 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:51:00.916408    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d\": container with ID starting with b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d not found: ID does not exist" containerID="b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"
Sep 10 17:51:00.916534 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.916434    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d"} err="failed to get container status \"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d\": rpc error: code = NotFound desc = could not find container \"b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d\": container with ID starting with b2ae8075d35509bd1d1e4f11e933a395aee62744f4312c699c40dfa28dde5c4d not found: ID does not exist"
Sep 10 17:51:00.937717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937694    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937818 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937730    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937818 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937749    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937818 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937770    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9w8j\" (UniqueName: \"kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937993 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937851    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937993 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937881    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs\") pod \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\" (UID: \"2871a189-2d25-484e-8c12-0f4ff2ba7f95\") "
Sep 10 17:51:00.937993 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937954    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:51:00.938114 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937989    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache" (OuterVolumeSpecName: "model-cache") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:51:00.938114 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.937993    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home" (OuterVolumeSpecName: "home") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:51:00.938186 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.938151    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:00.938186 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.938176    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:00.938257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.938193    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:00.939866 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.939846    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm" (OuterVolumeSpecName: "dshm") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:51:00.939960 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.939889    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:51:00.939960 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:00.939956    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j" (OuterVolumeSpecName: "kube-api-access-z9w8j") pod "2871a189-2d25-484e-8c12-0f4ff2ba7f95" (UID: "2871a189-2d25-484e-8c12-0f4ff2ba7f95"). InnerVolumeSpecName "kube-api-access-z9w8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:51:01.006986 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.006962    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" path="/var/lib/kubelet/pods/35c812c5-ba88-46a3-af00-97cbfb8cb1d6/volumes"
Sep 10 17:51:01.038706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.038685    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2871a189-2d25-484e-8c12-0f4ff2ba7f95-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:01.038706 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.038704    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9w8j\" (UniqueName: \"kubernetes.io/projected/2871a189-2d25-484e-8c12-0f4ff2ba7f95-kube-api-access-z9w8j\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:01.038828 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.038713    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2871a189-2d25-484e-8c12-0f4ff2ba7f95-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:51:01.226658 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.226584    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:51:01.235959 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:01.235940    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-7e6ec3a2-kserve-5d444d98f7-h6wvl"]
Sep 10 17:51:03.009227 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:03.009181    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" path="/var/lib/kubelet/pods/2871a189-2d25-484e-8c12-0f4ff2ba7f95/volumes"
Sep 10 17:51:04.144395 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:04.144349    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:51:09.698717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:09.698679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:18.922570 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:18.922502    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:21.683221 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:21.683184    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:51:26.117372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:26.117332    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:32.349869 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:32.349812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:32.462639 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:32.462607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:32.573633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:32.573607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:38.289782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:38.289744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:51:48.513580 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:48.513536    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:54.244392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:54.244355    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:51:54.638176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:54.638146    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:54.710755 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:54.710719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:54.801686 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:54.801654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:51:54.898751 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:51:54.898683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:00.194162 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194127    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194499    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" containerName="main"
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194512    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" containerName="main"
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194532    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" containerName="main"
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194538    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" containerName="main"
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194595    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2871a189-2d25-484e-8c12-0f4ff2ba7f95" containerName="main"
Sep 10 17:52:00.194633 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.194603    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c812c5-ba88-46a3-af00-97cbfb8cb1d6" containerName="main"
Sep 10 17:52:00.197626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.197604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207439 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207419    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c7995b\"/\"kube-root-ca.crt\""
Sep 10 17:52:00.207578 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207428    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c7995b\"/\"default-dockercfg-5bl2d\""
Sep 10 17:52:00.207578 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207512    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-c7995b\"/\"explicit-model-c7995b-kserve-self-signed-certs\""
Sep 10 17:52:00.207578 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207525    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-c7995b\"/\"openshift-service-ca.crt\""
Sep 10 17:52:00.207702 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207702 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207669    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59xsb\" (UniqueName: \"kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207729    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207756    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.207838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.207823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.225135 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.225111    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:00.245603 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.245575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:00.308923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.308900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.308929    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.308951    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.308968    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309026 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.309013    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309223 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.309035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59xsb\" (UniqueName: \"kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.309286    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309367 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.309343    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.309421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.309378    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.311353 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.311331    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.311611 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.311595    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.329226 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.329199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59xsb\" (UniqueName: \"kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb\") pod \"explicit-model-c7995b-kserve-6b7544b9ff-4hwdj\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") " pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.506723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.506658    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:00.781443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:00.781412    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:00.783998 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:52:00.783971    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98d85594_d1f8_4fc9_83f4_b04475f5a477.slice/crio-39bbbbe6a801a3108c7d782237fd70effe03ce4c370ed6b771a186e5dd3065fc WatchSource:0}: Error finding container 39bbbbe6a801a3108c7d782237fd70effe03ce4c370ed6b771a186e5dd3065fc: Status 404 returned error can't find the container with id 39bbbbe6a801a3108c7d782237fd70effe03ce4c370ed6b771a186e5dd3065fc
Sep 10 17:52:01.200807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:01.200769    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" event={"ID":"98d85594-d1f8-4fc9-83f4-b04475f5a477","Type":"ContainerStarted","Data":"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"}
Sep 10 17:52:01.200807 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:01.200813    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" event={"ID":"98d85594-d1f8-4fc9-83f4-b04475f5a477","Type":"ContainerStarted","Data":"39bbbbe6a801a3108c7d782237fd70effe03ce4c370ed6b771a186e5dd3065fc"}
Sep 10 17:52:01.222107 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:01.221948    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" podStartSLOduration=1.221931157 podStartE2EDuration="1.221931157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:01.221013407 +0000 UTC m=+936.738693570" watchObservedRunningTime="2026-09-10 17:52:01.221931157 +0000 UTC m=+936.739611320"
Sep 10 17:52:02.599907 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:02.599869    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:02.655646 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:02.655612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:05.395808 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:05.395768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:06.232096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:06.232055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:10.506798 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:10.506764    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:12.023333 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:12.023294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:12.138867 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:12.138828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:12.255188 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:12.255159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:12.351405 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:12.351329    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:13.107249 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:13.107216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:13.244431 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:13.244401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:13.674882 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:13.674857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:13.839502 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:13.839442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:13.937868 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:13.937805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:14.065331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:14.065285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:14.176916 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:14.176884    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:14.517817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:14.517790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:14.940298 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:14.940266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:15.174642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:15.174620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:15.273711 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:15.273650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:16.947042 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:16.947008    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:52:18.229961 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.229890    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" podUID="98d85594-d1f8-4fc9-83f4-b04475f5a477" containerName="main" containerID="cri-o://5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf" gracePeriod=45
Sep 10 17:52:18.478031 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.478009    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:18.543241 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543176    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543272    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543314    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543582 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543402    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543582 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543405    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home" (OuterVolumeSpecName: "home") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:18.543582 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543441    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59xsb\" (UniqueName: \"kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb\") pod \"98d85594-d1f8-4fc9-83f4-b04475f5a477\" (UID: \"98d85594-d1f8-4fc9-83f4-b04475f5a477\") "
Sep 10 17:52:18.543717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543675    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache" (OuterVolumeSpecName: "model-cache") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:18.543717 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543691    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:18.543804 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543787    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:18.543804 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543800    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:18.543881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.543812    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:18.545708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.545683    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb" (OuterVolumeSpecName: "kube-api-access-59xsb") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "kube-api-access-59xsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:18.545944 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.545915    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:52:18.546042 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.545917    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm" (OuterVolumeSpecName: "dshm") pod "98d85594-d1f8-4fc9-83f4-b04475f5a477" (UID: "98d85594-d1f8-4fc9-83f4-b04475f5a477"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:18.644930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.644912    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98d85594-d1f8-4fc9-83f4-b04475f5a477-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:18.644930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.644930    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98d85594-d1f8-4fc9-83f4-b04475f5a477-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:18.645067 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:18.644938    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59xsb\" (UniqueName: \"kubernetes.io/projected/98d85594-d1f8-4fc9-83f4-b04475f5a477-kube-api-access-59xsb\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:19.285355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.285288    2571 generic.go:356] "Generic (PLEG): container finished" podID="98d85594-d1f8-4fc9-83f4-b04475f5a477" containerID="5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf" exitCode=0
Sep 10 17:52:19.285355 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.285336    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"
Sep 10 17:52:19.285795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.285377    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" event={"ID":"98d85594-d1f8-4fc9-83f4-b04475f5a477","Type":"ContainerDied","Data":"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"}
Sep 10 17:52:19.285795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.285424    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj" event={"ID":"98d85594-d1f8-4fc9-83f4-b04475f5a477","Type":"ContainerDied","Data":"39bbbbe6a801a3108c7d782237fd70effe03ce4c370ed6b771a186e5dd3065fc"}
Sep 10 17:52:19.285795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.285443    2571 scope.go:117] "RemoveContainer" containerID="5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"
Sep 10 17:52:19.294192 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.294172    2571 scope.go:117] "RemoveContainer" containerID="5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"
Sep 10 17:52:19.294512 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:52:19.294484    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf\": container with ID starting with 5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf not found: ID does not exist" containerID="5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"
Sep 10 17:52:19.294636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.294516    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf"} err="failed to get container status \"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf\": rpc error: code = NotFound desc = could not find container \"5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf\": container with ID starting with 5a64a1de6d9c5fb35c2f7a50408df1dc9dabb0f665557700636501652f4a4acf not found: ID does not exist"
Sep 10 17:52:19.302965 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.302938    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:19.304622 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:19.304603    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-c7995b/explicit-model-c7995b-kserve-6b7544b9ff-4hwdj"]
Sep 10 17:52:20.876856 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:20.876814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:21.007598 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:21.007568    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d85594-d1f8-4fc9-83f4-b04475f5a477" path="/var/lib/kubelet/pods/98d85594-d1f8-4fc9-83f4-b04475f5a477/volumes"
Sep 10 17:52:21.062752 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:21.062723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:21.220885 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:21.220807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:21.678263 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:21.678236    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:22.009189 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:22.009160    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:22.200971 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:22.200943    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:22.495870 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:22.495831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:22.594366 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:22.594341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:23.219174 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:23.219135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:23.868079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:23.868042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:23.963214 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:23.963184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:24.085699 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:24.085666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:28.023429 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.023389    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:28.023837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.023763    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d85594-d1f8-4fc9-83f4-b04475f5a477" containerName="main"
Sep 10 17:52:28.023837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.023776    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d85594-d1f8-4fc9-83f4-b04475f5a477" containerName="main"
Sep 10 17:52:28.023837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.023833    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d85594-d1f8-4fc9-83f4-b04475f5a477" containerName="main"
Sep 10 17:52:28.028073 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.028057    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.030801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.030782    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8aeafa\"/\"openshift-service-ca.crt\""
Sep 10 17:52:28.031882 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.031856    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8aeafa\"/\"default-dockercfg-vgptn\""
Sep 10 17:52:28.031882 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.031857    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-8aeafa\"/\"noten-model-8aeafa-kserve-self-signed-certs\""
Sep 10 17:52:28.032072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.031886    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-8aeafa\"/\"kube-root-ca.crt\""
Sep 10 17:52:28.039742 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.039719    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:28.121684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121653    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.121684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.121881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.121881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121776    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.121881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlsvr\" (UniqueName: \"kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.121881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.121827    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.143236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.143192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:28.223251 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223208    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223251 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223318    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223340    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlsvr\" (UniqueName: \"kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223780 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223721    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223780 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223734    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.223850 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.223810    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.225814 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.225785    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.226115 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.226099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.232075 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.232056    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlsvr\" (UniqueName: \"kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr\") pod \"noten-model-8aeafa-kserve-675c8bb94c-mt29q\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") " pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.337585 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.337480    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:28.609883 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:28.609684    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:28.615241 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:52:28.612927    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacdeb787_e842_4496_9da9_2cae7082903f.slice/crio-d678eb4cb37232688d0f72409057a38b910423793677f797cdf72453b5827f7d WatchSource:0}: Error finding container d678eb4cb37232688d0f72409057a38b910423793677f797cdf72453b5827f7d: Status 404 returned error can't find the container with id d678eb4cb37232688d0f72409057a38b910423793677f797cdf72453b5827f7d
Sep 10 17:52:29.331058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:29.331021    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" event={"ID":"acdeb787-e842-4496-9da9-2cae7082903f","Type":"ContainerStarted","Data":"2c6b5017837be5d5a307998866204d3c57d4229a20911a7f3d02e8b058ebcd5d"}
Sep 10 17:52:29.331058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:29.331060    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" event={"ID":"acdeb787-e842-4496-9da9-2cae7082903f","Type":"ContainerStarted","Data":"d678eb4cb37232688d0f72409057a38b910423793677f797cdf72453b5827f7d"}
Sep 10 17:52:29.352925 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:29.352870    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" podStartSLOduration=1.352849413 podStartE2EDuration="1.352849413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:52:29.351176139 +0000 UTC m=+964.868856300" watchObservedRunningTime="2026-09-10 17:52:29.352849413 +0000 UTC m=+964.870529575"
Sep 10 17:52:30.013977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:30.013934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:30.118870 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:30.118835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:30.276604 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:30.276445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:30.669693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:30.669654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:31.212958 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:31.212922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:32.251936 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:32.251907    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:32.295207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:32.295179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:32.402753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:32.402720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:32.647265 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:32.647241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:32.768571 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:32.768540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:33.691654 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:33.691620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:33.781489 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:33.781441    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:34.743610 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:34.743580    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:52:35.039499 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:35.039404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:35.147638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:35.147598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:35.320316 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:35.320203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:35.528327 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:35.528299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:35.627487 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:35.627440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:37.858323 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:37.858289    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:38.337870 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:38.337839    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:39.321204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:39.321173    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:52:39.344424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:39.344380    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:40.298408 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:40.298375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:40.664433 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:40.664402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:47.270982 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.270898    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" podUID="acdeb787-e842-4496-9da9-2cae7082903f" containerName="main" containerID="cri-o://2c6b5017837be5d5a307998866204d3c57d4229a20911a7f3d02e8b058ebcd5d" gracePeriod=45
Sep 10 17:52:47.411264 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.411234    2571 generic.go:356] "Generic (PLEG): container finished" podID="acdeb787-e842-4496-9da9-2cae7082903f" containerID="2c6b5017837be5d5a307998866204d3c57d4229a20911a7f3d02e8b058ebcd5d" exitCode=0
Sep 10 17:52:47.411419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.411304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" event={"ID":"acdeb787-e842-4496-9da9-2cae7082903f","Type":"ContainerDied","Data":"2c6b5017837be5d5a307998866204d3c57d4229a20911a7f3d02e8b058ebcd5d"}
Sep 10 17:52:47.529043 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.528984    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:47.681632 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681597    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681641    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681682    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlsvr\" (UniqueName: \"kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681733    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681816    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681839    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs\") pod \"acdeb787-e842-4496-9da9-2cae7082903f\" (UID: \"acdeb787-e842-4496-9da9-2cae7082903f\") "
Sep 10 17:52:47.681995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681889    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache" (OuterVolumeSpecName: "model-cache") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:47.681995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.681925    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home" (OuterVolumeSpecName: "home") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:47.682216 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.682090    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:47.682216 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.682150    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:47.682216 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.682161    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:47.682216 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.682174    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:47.684128 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.684095    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:52:47.684237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.684165    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr" (OuterVolumeSpecName: "kube-api-access-xlsvr") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "kube-api-access-xlsvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:47.684237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.684203    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm" (OuterVolumeSpecName: "dshm") pod "acdeb787-e842-4496-9da9-2cae7082903f" (UID: "acdeb787-e842-4496-9da9-2cae7082903f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:47.782858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.782786    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdeb787-e842-4496-9da9-2cae7082903f-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:47.782858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.782821    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlsvr\" (UniqueName: \"kubernetes.io/projected/acdeb787-e842-4496-9da9-2cae7082903f-kube-api-access-xlsvr\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:47.782858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:47.782832    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdeb787-e842-4496-9da9-2cae7082903f-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:48.417302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:48.417275    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"
Sep 10 17:52:48.417721 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:48.417275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q" event={"ID":"acdeb787-e842-4496-9da9-2cae7082903f","Type":"ContainerDied","Data":"d678eb4cb37232688d0f72409057a38b910423793677f797cdf72453b5827f7d"}
Sep 10 17:52:48.417721 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:48.417410    2571 scope.go:117] "RemoveContainer" containerID="2c6b5017837be5d5a307998866204d3c57d4229a20911a7f3d02e8b058ebcd5d"
Sep 10 17:52:48.439862 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:48.439839    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:48.443359 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:48.443335    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-8aeafa/noten-model-8aeafa-kserve-675c8bb94c-mt29q"]
Sep 10 17:52:49.008204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:49.008175    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdeb787-e842-4496-9da9-2cae7082903f" path="/var/lib/kubelet/pods/acdeb787-e842-4496-9da9-2cae7082903f/volumes"
Sep 10 17:52:49.744774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:49.744745    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:52:50.947613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:50.947576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:51.121008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:51.120934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:51.233170 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:51.233057    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:51.332428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:51.332404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:51.405737 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:51.405694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:51.777054 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:51.777024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:54.341188 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.341152    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" podUID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" containerName="main" containerID="cri-o://2b430c36472d5e6576115b3a003c0f56b3aa5fad229afbfd4f2c6be122f268c2" gracePeriod=45
Sep 10 17:52:54.455443 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.455418    2571 generic.go:356] "Generic (PLEG): container finished" podID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" containerID="2b430c36472d5e6576115b3a003c0f56b3aa5fad229afbfd4f2c6be122f268c2" exitCode=0
Sep 10 17:52:54.455555 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.455490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" event={"ID":"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb","Type":"ContainerDied","Data":"2b430c36472d5e6576115b3a003c0f56b3aa5fad229afbfd4f2c6be122f268c2"}
Sep 10 17:52:54.583143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.583121    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:52:54.638837 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.638780    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.638949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.638871    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.638949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.638907    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz48w\" (UniqueName: \"kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.639025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.638951    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.639025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.638982    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.639126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.639023    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs\") pod \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\" (UID: \"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb\") "
Sep 10 17:52:54.639183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.639135    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home" (OuterVolumeSpecName: "home") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:54.639306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.639275    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:54.639421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.639300    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:54.639421 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.639358    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:54.641104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.641083    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm" (OuterVolumeSpecName: "dshm") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:52:54.641219 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.641197    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:52:54.641274 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.641228    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w" (OuterVolumeSpecName: "kube-api-access-lz48w") pod "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" (UID: "fef4c1cb-522d-47b3-aab5-1c37cb28b0bb"). InnerVolumeSpecName "kube-api-access-lz48w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:52:54.740146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.740121    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:54.740258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.740151    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:54.740258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.740163    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:54.740258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.740175    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lz48w\" (UniqueName: \"kubernetes.io/projected/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-kube-api-access-lz48w\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:54.740258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:54.740188    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:52:55.460917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.460897    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"
Sep 10 17:52:55.460917 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.460899    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz" event={"ID":"fef4c1cb-522d-47b3-aab5-1c37cb28b0bb","Type":"ContainerDied","Data":"166d2057ff22a1270c5c44cb56e453d57dbbb0973e613c691bb12679b16bd5ed"}
Sep 10 17:52:55.461370 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.460952    2571 scope.go:117] "RemoveContainer" containerID="2b430c36472d5e6576115b3a003c0f56b3aa5fad229afbfd4f2c6be122f268c2"
Sep 10 17:52:55.479124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.479100    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:52:55.483751 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.483731    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-48d00b/test-model-48d00b-kserve-7d68b97b9c-jp6jz"]
Sep 10 17:52:55.952809 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:55.952779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:57.007827 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:57.007787    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" path="/var/lib/kubelet/pods/fef4c1cb-522d-47b3-aab5-1c37cb28b0bb/volumes"
Sep 10 17:52:58.305112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:58.305075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:58.402341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:58.402304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:58.594640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:58.594568    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:58.730776 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:58.730743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:59.186294 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:59.186263    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:52:59.325736 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:59.325704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:59.578647 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:59.578569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:59.774424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:59.774395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:52:59.986838 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:52:59.986813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:05.870881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:05.870837    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:05.972053 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:05.972018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:06.507236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:06.507201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:09.143397 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:09.143365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:09.773172 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:09.773143    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:53:09.773348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:09.773328    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" podUID="e21f10f1-1384-4c74-9007-9d409f49b2b7" containerName="maas-api" containerID="cri-o://4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb" gracePeriod=30
Sep 10 17:53:09.811797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:09.811771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:09.913827 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:09.913799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:10.009718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.009698    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:53:10.038240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.038172    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:10.049549 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.049528    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xldng\" (UniqueName: \"kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng\") pod \"e21f10f1-1384-4c74-9007-9d409f49b2b7\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") "
Sep 10 17:53:10.049683 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.049600    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") pod \"e21f10f1-1384-4c74-9007-9d409f49b2b7\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") "
Sep 10 17:53:10.049683 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.049664    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs\") pod \"e21f10f1-1384-4c74-9007-9d409f49b2b7\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") "
Sep 10 17:53:10.049802 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.049705    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca\") pod \"e21f10f1-1384-4c74-9007-9d409f49b2b7\" (UID: \"e21f10f1-1384-4c74-9007-9d409f49b2b7\") "
Sep 10 17:53:10.050182 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.050116    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e21f10f1-1384-4c74-9007-9d409f49b2b7" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:53:10.051805 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.051769    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng" (OuterVolumeSpecName: "kube-api-access-xldng") pod "e21f10f1-1384-4c74-9007-9d409f49b2b7" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7"). InnerVolumeSpecName "kube-api-access-xldng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:53:10.051893 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.051799    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e21f10f1-1384-4c74-9007-9d409f49b2b7" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:53:10.051893 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.051845    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e21f10f1-1384-4c74-9007-9d409f49b2b7" (UID: "e21f10f1-1384-4c74-9007-9d409f49b2b7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:53:10.150427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.150400    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xldng\" (UniqueName: \"kubernetes.io/projected/e21f10f1-1384-4c74-9007-9d409f49b2b7-kube-api-access-xldng\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.150427 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.150423    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.150767 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.150432    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e21f10f1-1384-4c74-9007-9d409f49b2b7-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.150767 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.150440    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e21f10f1-1384-4c74-9007-9d409f49b2b7-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:53:10.186675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.186649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:10.499230 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.499203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:10.529034 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.529008    2571 generic.go:356] "Generic (PLEG): container finished" podID="e21f10f1-1384-4c74-9007-9d409f49b2b7" containerID="4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb" exitCode=0
Sep 10 17:53:10.529143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.529047    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"
Sep 10 17:53:10.529143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.529091    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" event={"ID":"e21f10f1-1384-4c74-9007-9d409f49b2b7","Type":"ContainerDied","Data":"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"}
Sep 10 17:53:10.529143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.529135    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t" event={"ID":"e21f10f1-1384-4c74-9007-9d409f49b2b7","Type":"ContainerDied","Data":"2cdcfc49c7caf51bfdbec46d48327b533c5bc6ebb4cb2bf1d8713b27405b6742"}
Sep 10 17:53:10.529265 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.529156    2571 scope.go:117] "RemoveContainer" containerID="4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"
Sep 10 17:53:10.542213 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.542193    2571 scope.go:117] "RemoveContainer" containerID="4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"
Sep 10 17:53:10.542522 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:53:10.542499    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb\": container with ID starting with 4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb not found: ID does not exist" containerID="4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"
Sep 10 17:53:10.542594 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.542535    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb"} err="failed to get container status \"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb\": rpc error: code = NotFound desc = could not find container \"4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb\": container with ID starting with 4581203ebfe479b4e38a9b959ced6d84012441138a9cbc9e0048783d31975ebb not found: ID does not exist"
Sep 10 17:53:10.555297 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.555275    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:53:10.559285 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:10.559266    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-783a5e-9f975dc8d-6t49t"]
Sep 10 17:53:11.007614 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:11.007582    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21f10f1-1384-4c74-9007-9d409f49b2b7" path="/var/lib/kubelet/pods/e21f10f1-1384-4c74-9007-9d409f49b2b7/volumes"
Sep 10 17:53:12.643630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:12.643598    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:53:15.482140 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:15.482107    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:16.546135 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:16.546099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.225909 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.225869    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.353705 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.353676    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.449857 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.449823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.546313 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.546203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.633240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.633212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:17.729589 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:17.729559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:18.046206 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:18.046173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:18.150009 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:18.149983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:18.633558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:18.633516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:19.161758 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:19.161727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:19.280044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:19.280018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:21.635630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:21.635599    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:53:21.813244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:21.813212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:21.915279 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:21.915252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:23.003336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:23.003305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:23.115251 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:23.115222    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:23.525015 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:23.524980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:23.926926 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:23.926895    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:24.137230 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:24.137202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:25.639789 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:25.639737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:25.740180 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:25.740127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:25.931659 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:25.931579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:26.365565 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:26.365439    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:26.496139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:26.496072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:26.621523 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:26.621415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:33.109904 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:33.109869    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:33.241903 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:33.241875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:33.408793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:33.408767    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:33.496098 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:33.496074    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:53:33.635385 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:33.635360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:39.745586 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:39.745554    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:39.842202 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:39.842173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:40.737663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:40.737630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:41.143797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:41.143762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:41.241765 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:41.241734    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:44.097469 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:44.097410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:44.193871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:44.193839    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:45.602184 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:45.602150    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:45.711475 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:45.711413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:45.895866 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:45.895825    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:46.177670 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:46.177590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:46.319832 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:46.319801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:46.865900 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:46.865852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:46.969043 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:46.969017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:47.246263 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:47.246151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:47.590272 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:47.590200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:47.688142 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:47.688110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:47.847158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:47.847076    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:49.517215 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:49.517182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:50.739142 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:50.739108    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:53:51.117641 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:51.117611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:51.215964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:51.215930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:52.333598 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333569    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333859    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdeb787-e842-4496-9da9-2cae7082903f" containerName="main"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333869    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdeb787-e842-4496-9da9-2cae7082903f" containerName="main"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333880    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" containerName="main"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333887    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" containerName="main"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333903    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e21f10f1-1384-4c74-9007-9d409f49b2b7" containerName="maas-api"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333909    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e21f10f1-1384-4c74-9007-9d409f49b2b7" containerName="maas-api"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333962    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdeb787-e842-4496-9da9-2cae7082903f" containerName="main"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333969    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e21f10f1-1384-4c74-9007-9d409f49b2b7" containerName="maas-api"
Sep 10 17:53:52.334006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.333976    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef4c1cb-522d-47b3-aab5-1c37cb28b0bb" containerName="main"
Sep 10 17:53:52.336786 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.336762    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.339420 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.339397    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 17:53:52.339558 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.339394    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-57474d-serving-cert\""
Sep 10 17:53:52.340580 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.340562    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 17:53:52.340708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.340628    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tdclw\""
Sep 10 17:53:52.340708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.340667    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 17:53:52.354161 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.354138    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 17:53:52.384246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.384214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:52.468344 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.468321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:52.475486 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.475444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d22d2\" (UniqueName: \"kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.475546 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.475530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.475586 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.475549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.475586 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.475567    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.576158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.576126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.576158 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.576161    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.576320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.576179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.576320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.576217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d22d2\" (UniqueName: \"kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.576320 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:53:52.576268    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-serving-cert: secret "maas-api-e2e-shared-b-w3-57474d-serving-cert" not found
Sep 10 17:53:52.576514 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:53:52.576335    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls podName:4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7 nodeName:}" failed. No retries permitted until 2026-09-10 17:53:53.07631952 +0000 UTC m=+1048.593999664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls") pod "maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" (UID: "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7") : secret "maas-api-e2e-shared-b-w3-57474d-serving-cert" not found
Sep 10 17:53:52.576902 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.576881    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.578754 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.578733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.587044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.586990    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d22d2\" (UniqueName: \"kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:52.924783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:52.924757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:53.080102 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.080075    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:53.082302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.082280    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:53.167975 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.167947    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:53.248070 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.248006    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:53.282104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.282079    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:53:53.541311 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.541286    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 17:53:53.543156 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:53:53.543131    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8ab0a4_3d8b_430b_8924_a5f8dfaec5a7.slice/crio-f1f5a1b5a2ba556f80f0386c6d77c1c95c97f26c868e25b904c10828b8605b52 WatchSource:0}: Error finding container f1f5a1b5a2ba556f80f0386c6d77c1c95c97f26c868e25b904c10828b8605b52: Status 404 returned error can't find the container with id f1f5a1b5a2ba556f80f0386c6d77c1c95c97f26c868e25b904c10828b8605b52
Sep 10 17:53:53.544676 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.544656    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 17:53:53.724072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:53.724044    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" event={"ID":"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7","Type":"ContainerStarted","Data":"f1f5a1b5a2ba556f80f0386c6d77c1c95c97f26c868e25b904c10828b8605b52"}
Sep 10 17:53:54.730344 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:54.730298    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" event={"ID":"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7","Type":"ContainerStarted","Data":"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"}
Sep 10 17:53:54.730739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:54.730442    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:53:54.749977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:53:54.749934    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" podStartSLOduration=2.424147521 podStartE2EDuration="2.749919811s" totalImagesPullingTime="325.77229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:53:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:53:54.74829636 +0000 UTC m=+1050.265976522" watchObservedRunningTime="2026-09-10 17:53:54.749919811 +0000 UTC m=+1050.267599970"
Sep 10 17:54:00.740762 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:00.740728    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 17:54:02.827613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:02.827578    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:54:06.942033 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:06.941985    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:09.185294 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:09.185253    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:09.565728 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:09.565654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:09.932211 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:09.932182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:10.087315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:10.087285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:13.355152 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:13.355119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:13.462348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:13.462317    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:13.812185 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:13.812157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:14.739471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.739419    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 17:54:14.742915 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.742896    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.745740 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.745716    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-bdf47c-serving-cert\""
Sep 10 17:54:14.753401 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.753376    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 17:54:14.842642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.842614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.842762 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.842646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.842762 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.842728    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7j8n\" (UniqueName: \"kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.842827 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.842761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.859964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.859939    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:14.943127 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.943102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.943235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.943134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.943235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.943194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7j8n\" (UniqueName: \"kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.943318 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.943233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.943359 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:14.943315    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-serving-cert: secret "maas-api-e2e-shared-b-w2-bdf47c-serving-cert" not found
Sep 10 17:54:14.943399 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:14.943387    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls podName:6678e15e-f73d-43e3-99f4-7623988e1c7e nodeName:}" failed. No retries permitted until 2026-09-10 17:54:15.443370658 +0000 UTC m=+1070.961050799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls") pod "maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" (UID: "6678e15e-f73d-43e3-99f4-7623988e1c7e") : secret "maas-api-e2e-shared-b-w2-bdf47c-serving-cert" not found
Sep 10 17:54:14.943977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.943957    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.945695 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.945666    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.957650 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.957631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7j8n\" (UniqueName: \"kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:14.971906 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:14.971881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:15.156057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.156037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:15.368762 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.368734    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:15.447650 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.447585    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:15.450024 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.449998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:15.471564 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.471537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:15.654620 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.654585    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:15.928928 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.928905    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 17:54:15.931200 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:15.931180    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6678e15e_f73d_43e3_99f4_7623988e1c7e.slice/crio-62a2424eaa669368c72531d7a064d1dc50029684aaa61eb628646c0318221261 WatchSource:0}: Error finding container 62a2424eaa669368c72531d7a064d1dc50029684aaa61eb628646c0318221261: Status 404 returned error can't find the container with id 62a2424eaa669368c72531d7a064d1dc50029684aaa61eb628646c0318221261
Sep 10 17:54:15.954117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:15.954066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:16.159044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:16.159020    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:16.834753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:16.834666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" event={"ID":"6678e15e-f73d-43e3-99f4-7623988e1c7e","Type":"ContainerStarted","Data":"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"}
Sep 10 17:54:16.834753 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:16.834742    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" event={"ID":"6678e15e-f73d-43e3-99f4-7623988e1c7e","Type":"ContainerStarted","Data":"62a2424eaa669368c72531d7a064d1dc50029684aaa61eb628646c0318221261"}
Sep 10 17:54:16.834939 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:16.834774    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:16.862701 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:16.862654    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" podStartSLOduration=2.366358212 podStartE2EDuration="2.862641967s" totalImagesPullingTime="496.283755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:16.861393768 +0000 UTC m=+1072.379073951" watchObservedRunningTime="2026-09-10 17:54:16.862641967 +0000 UTC m=+1072.380322155"
Sep 10 17:54:17.003403 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:17.003375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:19.756992 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:19.756957    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:54:22.844530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:22.844495    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 17:54:24.072839 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.072782    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:54:24.076124 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.076098    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.078775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.078751    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"openshift-service-ca.crt\""
Sep 10 17:54:24.078899 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.078770    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"kube-root-ca.crt\""
Sep 10 17:54:24.079896 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.079863    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"default-dockercfg-cfczq\""
Sep 10 17:54:24.080019 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.079919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"auth-test-model-18ecb2-kserve-self-signed-certs\""
Sep 10 17:54:24.097733 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.097711    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:54:24.116400 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.116547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116408    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.116547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116435    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9cq\" (UniqueName: \"kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.116547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116469    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.116684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.116684 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.116610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.134204 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.134180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:24.217428 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217547 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217529    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9cq\" (UniqueName: \"kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217733 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217840 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.217965 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217917    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.218038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.217998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.219964 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.219942    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.220146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.220131    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.226150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.226133    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9cq\" (UniqueName: \"kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq\") pod \"auth-test-model-18ecb2-kserve-55599bccf-bv44t\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.386392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.386371    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:24.656391 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.656306    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:54:24.659824 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:24.659793    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6860573f_a2ef_4318_afb4_8dee42c2c402.slice/crio-b458c3a96fd180138d6265934c100cb40d79b22c486cca0f49c5cb29b37cb424 WatchSource:0}: Error finding container b458c3a96fd180138d6265934c100cb40d79b22c486cca0f49c5cb29b37cb424: Status 404 returned error can't find the container with id b458c3a96fd180138d6265934c100cb40d79b22c486cca0f49c5cb29b37cb424
Sep 10 17:54:24.876273 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.876247    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" event={"ID":"6860573f-a2ef-4318-afb4-8dee42c2c402","Type":"ContainerStarted","Data":"11995eb3aaff0499fe8ff04e22c22905c687333cf1d81efe21cbc3bbb3d1016e"}
Sep 10 17:54:24.876418 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.876283    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" event={"ID":"6860573f-a2ef-4318-afb4-8dee42c2c402","Type":"ContainerStarted","Data":"b458c3a96fd180138d6265934c100cb40d79b22c486cca0f49c5cb29b37cb424"}
Sep 10 17:54:24.895819 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:24.895775    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" podStartSLOduration=0.895759103 podStartE2EDuration="895.759103ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:24.894966239 +0000 UTC m=+1080.412646401" watchObservedRunningTime="2026-09-10 17:54:24.895759103 +0000 UTC m=+1080.413439265"
Sep 10 17:54:34.386845 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:34.386803    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:34.387316 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:34.386872    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:34.399159 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:34.399131    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:34.530144 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:34.530116    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:54:34.934643 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:34.934612    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:54:38.345984 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:38.345944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:39.104691 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.104654    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-040273"]
Sep 10 17:54:39.108210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.108191    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:39.111110 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.111081    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w4zxj\""
Sep 10 17:54:39.112138 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.112104    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-040273"]
Sep 10 17:54:39.236923 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.236897    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx42x\" (UniqueName: \"kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x\") pod \"test-curl-7696-040273\" (UID: \"6f334b22-2060-4e43-8f3f-6eb82095c595\") " pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:39.338079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.338048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx42x\" (UniqueName: \"kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x\") pod \"test-curl-7696-040273\" (UID: \"6f334b22-2060-4e43-8f3f-6eb82095c595\") " pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:39.347048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.347019    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx42x\" (UniqueName: \"kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x\") pod \"test-curl-7696-040273\" (UID: \"6f334b22-2060-4e43-8f3f-6eb82095c595\") " pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:39.421593 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.421570    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:39.692689 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.692666    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-040273"]
Sep 10 17:54:39.695387 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:39.695361    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f334b22_2060_4e43_8f3f_6eb82095c595.slice/crio-a5dc21e6fe107d3df68b7327c5f0df9bd97da5f3f71274034955fd3b98e8cdfd WatchSource:0}: Error finding container a5dc21e6fe107d3df68b7327c5f0df9bd97da5f3f71274034955fd3b98e8cdfd: Status 404 returned error can't find the container with id a5dc21e6fe107d3df68b7327c5f0df9bd97da5f3f71274034955fd3b98e8cdfd
Sep 10 17:54:39.946635 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:39.946571    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-040273" event={"ID":"6f334b22-2060-4e43-8f3f-6eb82095c595","Type":"ContainerStarted","Data":"a5dc21e6fe107d3df68b7327c5f0df9bd97da5f3f71274034955fd3b98e8cdfd"}
Sep 10 17:54:41.958834 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:41.958796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-040273" event={"ID":"6f334b22-2060-4e43-8f3f-6eb82095c595","Type":"ContainerStarted","Data":"8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf"}
Sep 10 17:54:41.975721 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:41.975674    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-040273" podStartSLOduration=1.177660208 podStartE2EDuration="2.975657728s" totalImagesPullingTime="1.79799752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:41.973748054 +0000 UTC m=+1097.491428216" watchObservedRunningTime="2026-09-10 17:54:41.975657728 +0000 UTC m=+1097.493337890"
Sep 10 17:54:42.621892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.621861    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-040273"]
Sep 10 17:54:42.626510 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.626479    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-040273"]
Sep 10 17:54:42.748261 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.748195    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-fa89fc"]
Sep 10 17:54:42.748543 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.748530    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" containerName="test-curl-7696-040273"
Sep 10 17:54:42.748589 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.748544    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" containerName="test-curl-7696-040273"
Sep 10 17:54:42.748631 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.748595    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" containerName="test-curl-7696-040273"
Sep 10 17:54:42.751486 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.751445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:42.760739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.760718    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-fa89fc"]
Sep 10 17:54:42.868135 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.868111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9cxs\" (UniqueName: \"kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs\") pod \"test-curl-7696-fa89fc\" (UID: \"b00cd114-6bde-4bc6-9331-585c47e571db\") " pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:42.963888 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:42.963860    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-040273" container="test-curl-7696-040273"
Sep 10 17:54:42.969019 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.968997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9cxs\" (UniqueName: \"kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs\") pod \"test-curl-7696-fa89fc\" (UID: \"b00cd114-6bde-4bc6-9331-585c47e571db\") " pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:42.997110 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:42.997088    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9cxs\" (UniqueName: \"kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs\") pod \"test-curl-7696-fa89fc\" (UID: \"b00cd114-6bde-4bc6-9331-585c47e571db\") " pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:43.060118 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.060063    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:43.341630 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.341609    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-fa89fc"]
Sep 10 17:54:43.343292 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:43.343263    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb00cd114_6bde_4bc6_9331_585c47e571db.slice/crio-a3006e818e9ea604840549a3782e4b80299a939f08dbdd89e630af1f906d7ea4 WatchSource:0}: Error finding container a3006e818e9ea604840549a3782e4b80299a939f08dbdd89e630af1f906d7ea4: Status 404 returned error can't find the container with id a3006e818e9ea604840549a3782e4b80299a939f08dbdd89e630af1f906d7ea4
Sep 10 17:54:43.970653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.970614    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-fa89fc" event={"ID":"b00cd114-6bde-4bc6-9331-585c47e571db","Type":"ContainerStarted","Data":"35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb"}
Sep 10 17:54:43.971072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.970669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-fa89fc" event={"ID":"b00cd114-6bde-4bc6-9331-585c47e571db","Type":"ContainerStarted","Data":"a3006e818e9ea604840549a3782e4b80299a939f08dbdd89e630af1f906d7ea4"}
Sep 10 17:54:43.971072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.970762    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-040273" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" containerName="test-curl-7696-040273" containerID="cri-o://8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf" gracePeriod=2
Sep 10 17:54:43.997551 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:43.991878    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-fa89fc" podStartSLOduration=1.9918578070000001 podStartE2EDuration="1.991857807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:43.989437103 +0000 UTC m=+1099.507117264" watchObservedRunningTime="2026-09-10 17:54:43.991857807 +0000 UTC m=+1099.509537971"
Sep 10 17:54:44.618481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.618434    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-fa89fc"]
Sep 10 17:54:44.622062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.622032    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-fa89fc"]
Sep 10 17:54:44.736788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.736718    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-355285"]
Sep 10 17:54:44.737105 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.737091    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00cd114-6bde-4bc6-9331-585c47e571db" containerName="test-curl-7696-fa89fc"
Sep 10 17:54:44.737154 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.737107    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00cd114-6bde-4bc6-9331-585c47e571db" containerName="test-curl-7696-fa89fc"
Sep 10 17:54:44.737190 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.737171    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00cd114-6bde-4bc6-9331-585c47e571db" containerName="test-curl-7696-fa89fc"
Sep 10 17:54:44.740360 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.740344    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:44.743231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.743202    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-355285"]
Sep 10 17:54:44.785750 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.785725    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f665v\" (UniqueName: \"kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v\") pod \"test-curl-7696-355285\" (UID: \"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf\") " pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:44.886925 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.886897    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f665v\" (UniqueName: \"kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v\") pod \"test-curl-7696-355285\" (UID: \"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf\") " pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:44.895792 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:44.895767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f665v\" (UniqueName: \"kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v\") pod \"test-curl-7696-355285\" (UID: \"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf\") " pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:44.975721 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:44.975699    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-fa89fc" container="test-curl-7696-fa89fc"
Sep 10 17:54:45.050702 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:45.050651    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:45.326315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:45.326288    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-355285"]
Sep 10 17:54:45.327763 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:45.327737    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f2ef8d3_1e92_4dd9_affa_1a8ad3237cbf.slice/crio-b6838c68478753dc10ce309139c0489123c7320d351695ee133a783f4452b81e WatchSource:0}: Error finding container b6838c68478753dc10ce309139c0489123c7320d351695ee133a783f4452b81e: Status 404 returned error can't find the container with id b6838c68478753dc10ce309139c0489123c7320d351695ee133a783f4452b81e
Sep 10 17:54:45.982788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:45.982757    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-355285" event={"ID":"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf","Type":"ContainerStarted","Data":"a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7"}
Sep 10 17:54:45.983155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:45.982802    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-355285" event={"ID":"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf","Type":"ContainerStarted","Data":"b6838c68478753dc10ce309139c0489123c7320d351695ee133a783f4452b81e"}
Sep 10 17:54:45.983155 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:45.982852    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-fa89fc" podUID="b00cd114-6bde-4bc6-9331-585c47e571db" containerName="test-curl-7696-fa89fc" containerID="cri-o://35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb" gracePeriod=2
Sep 10 17:54:46.000840 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.000784    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-355285" podStartSLOduration=2.000766945 podStartE2EDuration="2.000766945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:46.00051763 +0000 UTC m=+1101.518197818" watchObservedRunningTime="2026-09-10 17:54:46.000766945 +0000 UTC m=+1101.518447130"
Sep 10 17:54:46.144761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.144739    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:46.196143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.196122    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx42x\" (UniqueName: \"kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x\") pod \"6f334b22-2060-4e43-8f3f-6eb82095c595\" (UID: \"6f334b22-2060-4e43-8f3f-6eb82095c595\") "
Sep 10 17:54:46.198374 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.198349    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x" (OuterVolumeSpecName: "kube-api-access-gx42x") pod "6f334b22-2060-4e43-8f3f-6eb82095c595" (UID: "6f334b22-2060-4e43-8f3f-6eb82095c595"). InnerVolumeSpecName "kube-api-access-gx42x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:54:46.296823 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.296771    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx42x\" (UniqueName: \"kubernetes.io/projected/6f334b22-2060-4e43-8f3f-6eb82095c595-kube-api-access-gx42x\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:54:46.608142 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.608079    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-355285"]
Sep 10 17:54:46.612793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.612769    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-355285"]
Sep 10 17:54:46.732587 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.732557    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-0bb123"]
Sep 10 17:54:46.732918 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.732904    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" containerName="test-curl-7696-355285"
Sep 10 17:54:46.732966 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.732920    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" containerName="test-curl-7696-355285"
Sep 10 17:54:46.733006 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.732973    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" containerName="test-curl-7696-355285"
Sep 10 17:54:46.736127 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.736111    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:46.741967 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.741941    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-0bb123"]
Sep 10 17:54:46.801218 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.801187    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzmlr\" (UniqueName: \"kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr\") pod \"test-curl-7696-0bb123\" (UID: \"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537\") " pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:46.901977 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.901955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzmlr\" (UniqueName: \"kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr\") pod \"test-curl-7696-0bb123\" (UID: \"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537\") " pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:46.910858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.910828    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzmlr\" (UniqueName: \"kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr\") pod \"test-curl-7696-0bb123\" (UID: \"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537\") " pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:46.988229 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.988197    2571 generic.go:356] "Generic (PLEG): container finished" podID="6f334b22-2060-4e43-8f3f-6eb82095c595" containerID="8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf" exitCode=137
Sep 10 17:54:46.988642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.988246    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:46.988642 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.988305    2571 scope.go:117] "RemoveContainer" containerID="8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf"
Sep 10 17:54:46.988642 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:46.988573    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-355285" container="test-curl-7696-355285"
Sep 10 17:54:46.997331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.997308    2571 scope.go:117] "RemoveContainer" containerID="8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf"
Sep 10 17:54:46.997684 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:46.997663    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf\": container with ID starting with 8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf not found: ID does not exist" containerID="8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf"
Sep 10 17:54:46.997770 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:46.997690    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf"} err="failed to get container status \"8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf\": rpc error: code = NotFound desc = could not find container \"8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf\": container with ID starting with 8bc39d114c3e8ec57c68b5019cbb0eb4c99c6092dd5150777a65d654e1036ddf not found: ID does not exist"
Sep 10 17:54:46.998445 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:46.998421    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-040273\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" pod="opendatahub/test-curl-7696-040273"
Sep 10 17:54:47.007518 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.007496    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f334b22-2060-4e43-8f3f-6eb82095c595" path="/var/lib/kubelet/pods/6f334b22-2060-4e43-8f3f-6eb82095c595/volumes"
Sep 10 17:54:47.046749 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.046733    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:47.327774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.327704    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-0bb123"]
Sep 10 17:54:47.331634 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:47.331605    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c7fd9aa_cedc_4aa8_8d9f_0c501a12f537.slice/crio-b6fecafa17a3364cc20e69bca27aa3b31ff12534c4068b95c9ed1e12000da4f9 WatchSource:0}: Error finding container b6fecafa17a3364cc20e69bca27aa3b31ff12534c4068b95c9ed1e12000da4f9: Status 404 returned error can't find the container with id b6fecafa17a3364cc20e69bca27aa3b31ff12534c4068b95c9ed1e12000da4f9
Sep 10 17:54:47.603820 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.603747    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:54:47.607398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.607382    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.610708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.610686    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"default-dockercfg-qfnb6\""
Sep 10 17:54:47.611584 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.611561    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"kube-root-ca.crt\""
Sep 10 17:54:47.611703 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.611631    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"auth-test-model-57474d-kserve-self-signed-certs\""
Sep 10 17:54:47.611703 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.611635    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"openshift-service-ca.crt\""
Sep 10 17:54:47.637821 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.637797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:54:47.696641 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.696606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:47.707775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707750    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.707874 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707779    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.707874 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.707958 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.707958 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707944    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2548\" (UniqueName: \"kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.708035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.707998    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809162 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809137    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2548\" (UniqueName: \"kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809187    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809535 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809307    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809590    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809705    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.809783 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.809720    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.811606 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.811587    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.812008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.811983    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.820576 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.820556    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2548\" (UniqueName: \"kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548\") pod \"auth-test-model-57474d-kserve-5975574c59-9m2tb\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") " pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.915834 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.915814    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:47.996430 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.996400    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-0bb123" event={"ID":"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537","Type":"ContainerStarted","Data":"89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e"}
Sep 10 17:54:47.996797 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.996446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-0bb123" event={"ID":"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537","Type":"ContainerStarted","Data":"b6fecafa17a3364cc20e69bca27aa3b31ff12534c4068b95c9ed1e12000da4f9"}
Sep 10 17:54:47.996987 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:47.996960    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-355285" podUID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" containerName="test-curl-7696-355285" containerID="cri-o://a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7" gracePeriod=2
Sep 10 17:54:48.016393 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.016350    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-0bb123" podStartSLOduration=2.016334724 podStartE2EDuration="2.016334724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:48.014419335 +0000 UTC m=+1103.532099499" watchObservedRunningTime="2026-09-10 17:54:48.016334724 +0000 UTC m=+1103.534014885"
Sep 10 17:54:48.149795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.149770    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:48.189725 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.189702    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:54:48.192614 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:54:48.192580    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0921db9_b781_4af7_bfbf_eae08a9ea9ba.slice/crio-9403fcd3c96d986c7ecf50803e3190998a92265ac647aad79d91386c98930f90 WatchSource:0}: Error finding container 9403fcd3c96d986c7ecf50803e3190998a92265ac647aad79d91386c98930f90: Status 404 returned error can't find the container with id 9403fcd3c96d986c7ecf50803e3190998a92265ac647aad79d91386c98930f90
Sep 10 17:54:48.213395 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.213373    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9cxs\" (UniqueName: \"kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs\") pod \"b00cd114-6bde-4bc6-9331-585c47e571db\" (UID: \"b00cd114-6bde-4bc6-9331-585c47e571db\") "
Sep 10 17:54:48.215550 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.215523    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs" (OuterVolumeSpecName: "kube-api-access-c9cxs") pod "b00cd114-6bde-4bc6-9331-585c47e571db" (UID: "b00cd114-6bde-4bc6-9331-585c47e571db"). InnerVolumeSpecName "kube-api-access-c9cxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:54:48.314230 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.314204    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9cxs\" (UniqueName: \"kubernetes.io/projected/b00cd114-6bde-4bc6-9331-585c47e571db-kube-api-access-c9cxs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:54:48.629839 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.629807    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-0bb123"]
Sep 10 17:54:48.634105 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:48.634083    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-0bb123"]
Sep 10 17:54:49.003327 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.003236    2571 generic.go:356] "Generic (PLEG): container finished" podID="b00cd114-6bde-4bc6-9331-585c47e571db" containerID="35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb" exitCode=137
Sep 10 17:54:49.003791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.003340    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-fa89fc"
Sep 10 17:54:49.005508 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:49.005481    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-0bb123" container="test-curl-7696-0bb123"
Sep 10 17:54:49.009892 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.009866    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00cd114-6bde-4bc6-9331-585c47e571db" path="/var/lib/kubelet/pods/b00cd114-6bde-4bc6-9331-585c47e571db/volumes"
Sep 10 17:54:49.010281 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.010265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" event={"ID":"c0921db9-b781-4af7-bfbf-eae08a9ea9ba","Type":"ContainerStarted","Data":"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"}
Sep 10 17:54:49.010335 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.010290    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" event={"ID":"c0921db9-b781-4af7-bfbf-eae08a9ea9ba","Type":"ContainerStarted","Data":"9403fcd3c96d986c7ecf50803e3190998a92265ac647aad79d91386c98930f90"}
Sep 10 17:54:49.010392 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.010333    2571 scope.go:117] "RemoveContainer" containerID="35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb"
Sep 10 17:54:49.019492 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.019447    2571 scope.go:117] "RemoveContainer" containerID="35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb"
Sep 10 17:54:49.019773 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:49.019755    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb\": container with ID starting with 35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb not found: ID does not exist" containerID="35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb"
Sep 10 17:54:49.019855 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.019784    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb"} err="failed to get container status \"35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb\": rpc error: code = NotFound desc = could not find container \"35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb\": container with ID starting with 35cba0aac40309bd4ccbe4bdbd9b9723035f581a582045cefa20f5f82f2ffcfb not found: ID does not exist"
Sep 10 17:54:49.023108 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.023070    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" podStartSLOduration=2.023057729 podStartE2EDuration="2.023057729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:54:49.021431189 +0000 UTC m=+1104.539111350" watchObservedRunningTime="2026-09-10 17:54:49.023057729 +0000 UTC m=+1104.540737890"
Sep 10 17:54:49.356846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.356782    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:49.633522 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.633497    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:49.909104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:49.909040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:50.010886 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.010861    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-0bb123" podUID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" containerName="test-curl-7696-0bb123" containerID="cri-o://89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e" gracePeriod=2
Sep 10 17:54:50.184728 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.184706    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:50.330627 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.330599    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f665v\" (UniqueName: \"kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v\") pod \"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf\" (UID: \"7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf\") "
Sep 10 17:54:50.332825 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.332794    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v" (OuterVolumeSpecName: "kube-api-access-f665v") pod "7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" (UID: "7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf"). InnerVolumeSpecName "kube-api-access-f665v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:54:50.431792 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.431736    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f665v\" (UniqueName: \"kubernetes.io/projected/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf-kube-api-access-f665v\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:54:50.672020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:50.671991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:51.008819 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.008783    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" path="/var/lib/kubelet/pods/7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf/volumes"
Sep 10 17:54:51.015846 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.015818    2571 generic.go:356] "Generic (PLEG): container finished" podID="7f2ef8d3-1e92-4dd9-affa-1a8ad3237cbf" containerID="a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7" exitCode=137
Sep 10 17:54:51.016236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.015878    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-355285"
Sep 10 17:54:51.016236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.015896    2571 scope.go:117] "RemoveContainer" containerID="a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7"
Sep 10 17:54:51.025240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.025221    2571 scope.go:117] "RemoveContainer" containerID="a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7"
Sep 10 17:54:51.025546 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:51.025525    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7\": container with ID starting with a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7 not found: ID does not exist" containerID="a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7"
Sep 10 17:54:51.025610 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:51.025561    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7"} err="failed to get container status \"a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7\": rpc error: code = NotFound desc = could not find container \"a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7\": container with ID starting with a59c314c5d09abc02b3569ed285850132c575783607ee1850b0d38eaea2240f7 not found: ID does not exist"
Sep 10 17:54:52.153665 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.153643    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:52.245930 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.245903    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzmlr\" (UniqueName: \"kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr\") pod \"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537\" (UID: \"5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537\") "
Sep 10 17:54:52.247858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.247836    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr" (OuterVolumeSpecName: "kube-api-access-vzmlr") pod "5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" (UID: "5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537"). InnerVolumeSpecName "kube-api-access-vzmlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:54:52.338160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.338101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:52.347321 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.347295    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzmlr\" (UniqueName: \"kubernetes.io/projected/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537-kube-api-access-vzmlr\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:54:52.434653 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:52.434629    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:53.007388 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.007359    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" path="/var/lib/kubelet/pods/5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537/volumes"
Sep 10 17:54:53.029032 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.028999    2571 generic.go:356] "Generic (PLEG): container finished" podID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" containerID="89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e" exitCode=137
Sep 10 17:54:53.029156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.029038    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-0bb123"
Sep 10 17:54:53.029156 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.029081    2571 scope.go:117] "RemoveContainer" containerID="89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e"
Sep 10 17:54:53.037560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.037541    2571 scope.go:117] "RemoveContainer" containerID="89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e"
Sep 10 17:54:53.037862 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:54:53.037842    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e\": container with ID starting with 89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e not found: ID does not exist" containerID="89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e"
Sep 10 17:54:53.037927 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.037868    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e"} err="failed to get container status \"89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e\": rpc error: code = NotFound desc = could not find container \"89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e\": container with ID starting with 89402e4103acefdb0cf2cc002faaeeb899ede188de1277a9469b892108b8068e not found: ID does not exist"
Sep 10 17:54:53.437012 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.436981    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:54:53.515516 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.515486    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:53.617490 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:53.617428    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:54.135278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:54.135242    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:54.290223 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:54.290190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:54.425378 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:54.425280    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:54:57.916057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:57.916013    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:57.916057 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:57.916068    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:57.932721 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:57.932694    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:54:58.071330 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:54:58.071297    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:55:11.389757 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:11.389715    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:55:15.076959 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:15.076922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:55:17.313962 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:17.313917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:55:21.247349 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.247315    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:55:21.247761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.247653    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" containerName="test-curl-7696-0bb123"
Sep 10 17:55:21.247761 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.247695    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" containerName="test-curl-7696-0bb123"
Sep 10 17:55:21.247840 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.247785    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c7fd9aa-cedc-4aa8-8d9f-0c501a12f537" containerName="test-curl-7696-0bb123"
Sep 10 17:55:21.252895 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.252873    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.255692 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.255670    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-655f7f0b\"/\"default-dockercfg-w4mn6\""
Sep 10 17:55:21.255813 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.255744    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-655f7f0b\"/\"kube-root-ca.crt\""
Sep 10 17:55:21.256663 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.256631    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-655f7f0b\"/\"e2e-lifecycle-model-655f7f0b-kserve-self-signed-certs\""
Sep 10 17:55:21.256830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.256680    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-655f7f0b\"/\"openshift-service-ca.crt\""
Sep 10 17:55:21.263096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.263074    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:55:21.377434 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7t8r\" (UniqueName: \"kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.377566 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.377566 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.377667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.377667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.377667 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.377617    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.451658 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.451626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:55:21.478196 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7t8r\" (UniqueName: \"kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478277    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478325 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478522 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478522 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478357    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478642    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478706    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.478754 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.478736    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.480675 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.480655    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.480942 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.480926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.489909 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.489883    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7t8r\" (UniqueName: \"kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r\") pod \"e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") " pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.562926 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.562868    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:21.838257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:21.838233    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:55:21.840421 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:55:21.840386    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd24593f2_88cb_408f_8f22_6c11c7eee07b.slice/crio-0d00edb048e623e9f5b262bc7c0706e1b934122f99f0914d51846e2a34d70311 WatchSource:0}: Error finding container 0d00edb048e623e9f5b262bc7c0706e1b934122f99f0914d51846e2a34d70311: Status 404 returned error can't find the container with id 0d00edb048e623e9f5b262bc7c0706e1b934122f99f0914d51846e2a34d70311
Sep 10 17:55:22.177512 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:22.177480    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" event={"ID":"d24593f2-88cb-408f-8f22-6c11c7eee07b","Type":"ContainerStarted","Data":"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"}
Sep 10 17:55:22.177666 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:22.177524    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" event={"ID":"d24593f2-88cb-408f-8f22-6c11c7eee07b","Type":"ContainerStarted","Data":"0d00edb048e623e9f5b262bc7c0706e1b934122f99f0914d51846e2a34d70311"}
Sep 10 17:55:22.196872 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:22.196826    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" podStartSLOduration=1.196810579 podStartE2EDuration="1.196810579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:55:22.195036019 +0000 UTC m=+1137.712716182" watchObservedRunningTime="2026-09-10 17:55:22.196810579 +0000 UTC m=+1137.714490754"
Sep 10 17:55:27.332471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:27.332422    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:55:31.563150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:31.563113    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:31.563150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:31.563151    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:31.576117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:31.576092    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:32.241760 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:32.241732    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:55:44.934192 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:44.934161    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:55:50.310195 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:55:50.310115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:03.751713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:03.751669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:56:07.193343 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:07.193301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:07.293234 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:07.293194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:07.392815 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:07.392787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:07.454058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:07.453985    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:10.574712 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:10.574679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:15.551123 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:15.551079    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:56:15.611080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:15.611046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:19.642348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:19.642314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:56:30.572471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.572376    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" podUID="d24593f2-88cb-408f-8f22-6c11c7eee07b" containerName="main" containerID="cri-o://03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4" gracePeriod=45
Sep 10 17:56:30.827108 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.827055    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:56:30.835354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835335    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835378    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835400    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7t8r\" (UniqueName: \"kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835414    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835443    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835490    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home\") pod \"d24593f2-88cb-408f-8f22-6c11c7eee07b\" (UID: \"d24593f2-88cb-408f-8f22-6c11c7eee07b\") "
Sep 10 17:56:30.835723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835665    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:56:30.835723 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835704    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache" (OuterVolumeSpecName: "model-cache") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:56:30.835879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835781    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home" (OuterVolumeSpecName: "home") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:56:30.835879 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835871    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.835985 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.835890    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.837670 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.837645    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm" (OuterVolumeSpecName: "dshm") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:56:30.837782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.837707    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:56:30.837849 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.837812    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r" (OuterVolumeSpecName: "kube-api-access-p7t8r") pod "d24593f2-88cb-408f-8f22-6c11c7eee07b" (UID: "d24593f2-88cb-408f-8f22-6c11c7eee07b"). InnerVolumeSpecName "kube-api-access-p7t8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:56:30.936685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.936649    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.936685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.936679    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d24593f2-88cb-408f-8f22-6c11c7eee07b-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.936862 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.936691    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d24593f2-88cb-408f-8f22-6c11c7eee07b-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:30.936862 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:30.936706    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7t8r\" (UniqueName: \"kubernetes.io/projected/d24593f2-88cb-408f-8f22-6c11c7eee07b-kube-api-access-p7t8r\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:31.520301 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.520269    2571 generic.go:356] "Generic (PLEG): container finished" podID="d24593f2-88cb-408f-8f22-6c11c7eee07b" containerID="03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4" exitCode=0
Sep 10 17:56:31.520480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.520314    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"
Sep 10 17:56:31.520480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.520351    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" event={"ID":"d24593f2-88cb-408f-8f22-6c11c7eee07b","Type":"ContainerDied","Data":"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"}
Sep 10 17:56:31.520480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.520396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr" event={"ID":"d24593f2-88cb-408f-8f22-6c11c7eee07b","Type":"ContainerDied","Data":"0d00edb048e623e9f5b262bc7c0706e1b934122f99f0914d51846e2a34d70311"}
Sep 10 17:56:31.520480 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.520416    2571 scope.go:117] "RemoveContainer" containerID="03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"
Sep 10 17:56:31.529231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.529213    2571 scope.go:117] "RemoveContainer" containerID="03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"
Sep 10 17:56:31.529537 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:56:31.529519    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4\": container with ID starting with 03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4 not found: ID does not exist" containerID="03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"
Sep 10 17:56:31.529596 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.529544    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4"} err="failed to get container status \"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4\": rpc error: code = NotFound desc = could not find container \"03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4\": container with ID starting with 03dd484b69f7134bbd6aea30767e4b04f69666c1ad9febc503e11c88506ac8c4 not found: ID does not exist"
Sep 10 17:56:31.545686 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.545661    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:56:31.549064 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:31.549044    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-655f7f0b/e2e-lifecycle-model-655f7f0b-kserve-b46dd7647-8dkqr"]
Sep 10 17:56:33.007258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:33.007225    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d24593f2-88cb-408f-8f22-6c11c7eee07b" path="/var/lib/kubelet/pods/d24593f2-88cb-408f-8f22-6c11c7eee07b/volumes"
Sep 10 17:56:34.277296 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:34.277251    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:35.635010 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:35.634974    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:56:49.854176 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:49.854130    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:56:50.820896 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.820864    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7699-7d3bda"]
Sep 10 17:56:50.821290 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.821273    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d24593f2-88cb-408f-8f22-6c11c7eee07b" containerName="main"
Sep 10 17:56:50.821386 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.821294    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d24593f2-88cb-408f-8f22-6c11c7eee07b" containerName="main"
Sep 10 17:56:50.821386 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.821380    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d24593f2-88cb-408f-8f22-6c11c7eee07b" containerName="main"
Sep 10 17:56:50.824563 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.824535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:50.827309 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.827284    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w4zxj\""
Sep 10 17:56:50.829726 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.829704    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-7d3bda"]
Sep 10 17:56:50.893349 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.893325    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdt2\" (UniqueName: \"kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2\") pod \"test-curl-7699-7d3bda\" (UID: \"d3851200-1897-416b-9daf-7794b00576bd\") " pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:50.993856 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:50.993830    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdt2\" (UniqueName: \"kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2\") pod \"test-curl-7699-7d3bda\" (UID: \"d3851200-1897-416b-9daf-7794b00576bd\") " pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:51.002625 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.002597    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdt2\" (UniqueName: \"kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2\") pod \"test-curl-7699-7d3bda\" (UID: \"d3851200-1897-416b-9daf-7794b00576bd\") " pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:51.135517 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.135492    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:51.414708 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.414669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-7d3bda"]
Sep 10 17:56:51.415385 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:56:51.415351    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3851200_1897_416b_9daf_7794b00576bd.slice/crio-0ed96e1ee9a2548ad646a6f6b51906246494faf992fd0a58676058906e968d1e WatchSource:0}: Error finding container 0ed96e1ee9a2548ad646a6f6b51906246494faf992fd0a58676058906e968d1e: Status 404 returned error can't find the container with id 0ed96e1ee9a2548ad646a6f6b51906246494faf992fd0a58676058906e968d1e
Sep 10 17:56:51.615499 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.615471    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-7d3bda" event={"ID":"d3851200-1897-416b-9daf-7794b00576bd","Type":"ContainerStarted","Data":"0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760"}
Sep 10 17:56:51.615637 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.615512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-7d3bda" event={"ID":"d3851200-1897-416b-9daf-7794b00576bd","Type":"ContainerStarted","Data":"0ed96e1ee9a2548ad646a6f6b51906246494faf992fd0a58676058906e968d1e"}
Sep 10 17:56:51.632147 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:51.632102    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7699-7d3bda" podStartSLOduration=1.632086084 podStartE2EDuration="1.632086084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:56:51.629935838 +0000 UTC m=+1227.147616000" watchObservedRunningTime="2026-09-10 17:56:51.632086084 +0000 UTC m=+1227.149766245"
Sep 10 17:56:52.068817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:52.068751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:52.224184 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:52.224155    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7699-7d3bda"]
Sep 10 17:56:52.228344 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:52.228318    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7699-7d3bda"]
Sep 10 17:56:52.619946 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:56:52.619915    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7699-7d3bda" container="test-curl-7699-7d3bda"
Sep 10 17:56:53.237873 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:53.237838    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:53.567264 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:53.567190    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:56:53.595608 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:53.595576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:53.624345 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:53.624307    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7699-7d3bda" podUID="d3851200-1897-416b-9daf-7794b00576bd" containerName="test-curl-7699-7d3bda" containerID="cri-o://0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760" gracePeriod=2
Sep 10 17:56:54.650900 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:54.650868    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:56:54.686055 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:54.686029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:55.793084 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:55.793059    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:55.935956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:55.935894    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmdt2\" (UniqueName: \"kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2\") pod \"d3851200-1897-416b-9daf-7794b00576bd\" (UID: \"d3851200-1897-416b-9daf-7794b00576bd\") "
Sep 10 17:56:55.938079 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:55.938048    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2" (OuterVolumeSpecName: "kube-api-access-bmdt2") pod "d3851200-1897-416b-9daf-7794b00576bd" (UID: "d3851200-1897-416b-9daf-7794b00576bd"). InnerVolumeSpecName "kube-api-access-bmdt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:56:56.036990 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.036959    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmdt2\" (UniqueName: \"kubernetes.io/projected/d3851200-1897-416b-9daf-7794b00576bd-kube-api-access-bmdt2\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:56:56.642468 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.642421    2571 generic.go:356] "Generic (PLEG): container finished" podID="d3851200-1897-416b-9daf-7794b00576bd" containerID="0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760" exitCode=137
Sep 10 17:56:56.642636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.642494    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:56.642636 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.642540    2571 scope.go:117] "RemoveContainer" containerID="0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760"
Sep 10 17:56:56.652089 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.652068    2571 scope.go:117] "RemoveContainer" containerID="0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760"
Sep 10 17:56:56.652378 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:56:56.652358    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760\": container with ID starting with 0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760 not found: ID does not exist" containerID="0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760"
Sep 10 17:56:56.652429 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:56.652392    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760"} err="failed to get container status \"0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760\": rpc error: code = NotFound desc = could not find container \"0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760\": container with ID starting with 0c069c649a0de967fd5643d4c89283251163b116a6644687f4589c9f798ab760 not found: ID does not exist"
Sep 10 17:56:56.652686 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:56:56.652661    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7699-7d3bda\" is forbidden: User \"system:node:ip-10-0-140-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-165.ec2.internal' and this object" podUID="d3851200-1897-416b-9daf-7794b00576bd" pod="opendatahub/test-curl-7699-7d3bda"
Sep 10 17:56:57.007120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:57.007058    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3851200-1897-416b-9daf-7794b00576bd" path="/var/lib/kubelet/pods/d3851200-1897-416b-9daf-7794b00576bd/volumes"
Sep 10 17:56:59.021161 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:59.021130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:59.099949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:59.099892    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:56:59.288183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:56:59.288112    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:00.657910 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:00.657881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:00.777080 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:00.777045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:00.876415 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:00.876375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:01.507409 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:01.507379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:01.602733 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:01.602712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:02.999883 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:02.999848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:04.651258 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.651215    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:57:04.651801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.651574    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3851200-1897-416b-9daf-7794b00576bd" containerName="test-curl-7699-7d3bda"
Sep 10 17:57:04.651801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.651587    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3851200-1897-416b-9daf-7794b00576bd" containerName="test-curl-7699-7d3bda"
Sep 10 17:57:04.651801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.651659    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3851200-1897-416b-9daf-7794b00576bd" containerName="test-curl-7699-7d3bda"
Sep 10 17:57:04.656126 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.656093    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.658709 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.658685    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"sub-test-model-18ecb2-kserve-self-signed-certs\""
Sep 10 17:57:04.664782 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.664757    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:57:04.692711 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.692683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:04.813900 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.813878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.814018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.813909    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnvz4\" (UniqueName: \"kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.814018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.813937    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.814018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.814007    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.814139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.814064    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.814139 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.814081    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915205 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915149    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915205 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915255    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnvz4\" (UniqueName: \"kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915291    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915644 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915622    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915628    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.915774 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.915743    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.917749 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.917721    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.917989 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.917973    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.924867 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.924846    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnvz4\" (UniqueName: \"kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4\") pod \"sub-test-model-18ecb2-kserve-66b75889df-9w7l8\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:04.967719 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:04.967691    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:05.242864 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:05.242797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:57:05.245671 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:57:05.245639    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda89f8e1d_1d61_4630_89e1_d530a363574f.slice/crio-193b9ad7532cd73d04f8d12080d9886578640010ee23cdb668d7150650925c14 WatchSource:0}: Error finding container 193b9ad7532cd73d04f8d12080d9886578640010ee23cdb668d7150650925c14: Status 404 returned error can't find the container with id 193b9ad7532cd73d04f8d12080d9886578640010ee23cdb668d7150650925c14
Sep 10 17:57:05.437561 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:05.437526    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:57:05.688217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:05.688178    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" event={"ID":"a89f8e1d-1d61-4630-89e1-d530a363574f","Type":"ContainerStarted","Data":"72811736e8309002381d6c70635b299b1f14aac59951b5ff9198f529e66528f5"}
Sep 10 17:57:05.688629 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:05.688227    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" event={"ID":"a89f8e1d-1d61-4630-89e1-d530a363574f","Type":"ContainerStarted","Data":"193b9ad7532cd73d04f8d12080d9886578640010ee23cdb668d7150650925c14"}
Sep 10 17:57:05.710251 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:05.710178    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" podStartSLOduration=1.710157973 podStartE2EDuration="1.710157973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:05.709359826 +0000 UTC m=+1241.227039988" watchObservedRunningTime="2026-09-10 17:57:05.710157973 +0000 UTC m=+1241.227838179"
Sep 10 17:57:08.587326 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.587266    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" podUID="6860573f-a2ef-4318-afb4-8dee42c2c402" containerName="main" containerID="cri-o://11995eb3aaff0499fe8ff04e22c22905c687333cf1d81efe21cbc3bbb3d1016e" gracePeriod=45
Sep 10 17:57:08.705956 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.705905    2571 generic.go:356] "Generic (PLEG): container finished" podID="6860573f-a2ef-4318-afb4-8dee42c2c402" containerID="11995eb3aaff0499fe8ff04e22c22905c687333cf1d81efe21cbc3bbb3d1016e" exitCode=0
Sep 10 17:57:08.706025 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.705972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" event={"ID":"6860573f-a2ef-4318-afb4-8dee42c2c402","Type":"ContainerDied","Data":"11995eb3aaff0499fe8ff04e22c22905c687333cf1d81efe21cbc3bbb3d1016e"}
Sep 10 17:57:08.834267 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.834241    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:57:08.912113 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.912058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:08.948095 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948070    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r9cq\" (UniqueName: \"kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948103    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948145    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948159    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948217 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948178    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948386 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948260    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs\") pod \"6860573f-a2ef-4318-afb4-8dee42c2c402\" (UID: \"6860573f-a2ef-4318-afb4-8dee42c2c402\") "
Sep 10 17:57:08.948444 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948410    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:08.948540 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948482    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache" (OuterVolumeSpecName: "model-cache") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:08.948613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948547    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home" (OuterVolumeSpecName: "home") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:08.948613 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.948563    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:08.950588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.950559    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:08.950588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.950571    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq" (OuterVolumeSpecName: "kube-api-access-9r9cq") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "kube-api-access-9r9cq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:57:08.950736 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:08.950569    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm" (OuterVolumeSpecName: "dshm") pod "6860573f-a2ef-4318-afb4-8dee42c2c402" (UID: "6860573f-a2ef-4318-afb4-8dee42c2c402"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:09.011544 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.011522    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:09.049612 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.049587    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6860573f-a2ef-4318-afb4-8dee42c2c402-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:09.049612 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.049612    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r9cq\" (UniqueName: \"kubernetes.io/projected/6860573f-a2ef-4318-afb4-8dee42c2c402-kube-api-access-9r9cq\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:09.049735 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.049622    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:09.049735 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.049630    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:09.049735 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.049639    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6860573f-a2ef-4318-afb4-8dee42c2c402-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:09.626691 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.626656    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:09.670886 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.670838    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" podUID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" containerName="main" containerID="cri-o://ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450" gracePeriod=45
Sep 10 17:57:09.712038 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.712017    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"
Sep 10 17:57:09.712146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.712022    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t" event={"ID":"6860573f-a2ef-4318-afb4-8dee42c2c402","Type":"ContainerDied","Data":"b458c3a96fd180138d6265934c100cb40d79b22c486cca0f49c5cb29b37cb424"}
Sep 10 17:57:09.712146 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.712137    2571 scope.go:117] "RemoveContainer" containerID="11995eb3aaff0499fe8ff04e22c22905c687333cf1d81efe21cbc3bbb3d1016e"
Sep 10 17:57:09.756491 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.756445    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:57:09.761884 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.761864    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/auth-test-model-18ecb2-kserve-55599bccf-bv44t"]
Sep 10 17:57:09.910834 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:09.910810    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:57:10.057302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057274    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057310    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057330    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057343    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057438 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057432    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057496    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2548\" (UniqueName: \"kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548\") pod \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\" (UID: \"c0921db9-b781-4af7-bfbf-eae08a9ea9ba\") "
Sep 10 17:57:10.057638 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057582    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:10.057747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057667    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:10.057747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057703    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home" (OuterVolumeSpecName: "home") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:10.057858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057824    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.057858 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057848    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.057950 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.057863    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.059725 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.059702    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm" (OuterVolumeSpecName: "dshm") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:57:10.059841 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.059734    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:57:10.059841 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.059761    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548" (OuterVolumeSpecName: "kube-api-access-q2548") pod "c0921db9-b781-4af7-bfbf-eae08a9ea9ba" (UID: "c0921db9-b781-4af7-bfbf-eae08a9ea9ba"). InnerVolumeSpecName "kube-api-access-q2548". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:57:10.158974 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.158923    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2548\" (UniqueName: \"kubernetes.io/projected/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-kube-api-access-q2548\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.158974 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.158943    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.158974 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.158953    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0921db9-b781-4af7-bfbf-eae08a9ea9ba-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:57:10.720326 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.720290    2571 generic.go:356] "Generic (PLEG): container finished" podID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" containerID="ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450" exitCode=0
Sep 10 17:57:10.720779 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.720341    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"
Sep 10 17:57:10.720779 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.720359    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" event={"ID":"c0921db9-b781-4af7-bfbf-eae08a9ea9ba","Type":"ContainerDied","Data":"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"}
Sep 10 17:57:10.720779 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.720405    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb" event={"ID":"c0921db9-b781-4af7-bfbf-eae08a9ea9ba","Type":"ContainerDied","Data":"9403fcd3c96d986c7ecf50803e3190998a92265ac647aad79d91386c98930f90"}
Sep 10 17:57:10.720779 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.720429    2571 scope.go:117] "RemoveContainer" containerID="ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"
Sep 10 17:57:10.730416 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.730396    2571 scope.go:117] "RemoveContainer" containerID="ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"
Sep 10 17:57:10.730764 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:57:10.730741    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450\": container with ID starting with ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450 not found: ID does not exist" containerID="ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"
Sep 10 17:57:10.730823 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.730773    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450"} err="failed to get container status \"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450\": rpc error: code = NotFound desc = could not find container \"ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450\": container with ID starting with ac6ad4ecb17ab7b082222c6171e24c807d1869d255efabea9e5007894aa5e450 not found: ID does not exist"
Sep 10 17:57:10.742008 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.741980    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:57:10.746394 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:10.746372    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/auth-test-model-57474d-kserve-5975574c59-9m2tb"]
Sep 10 17:57:11.012972 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:11.012896    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6860573f-a2ef-4318-afb4-8dee42c2c402" path="/var/lib/kubelet/pods/6860573f-a2ef-4318-afb4-8dee42c2c402/volumes"
Sep 10 17:57:11.013270 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:11.013257    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" path="/var/lib/kubelet/pods/c0921db9-b781-4af7-bfbf-eae08a9ea9ba/volumes"
Sep 10 17:57:14.968100 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:14.968063    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:14.968521 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:14.968116    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:14.981169 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:14.981147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:15.758172 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:15.758138    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:57:16.002042 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:16.001972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:16.100553 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:16.100487    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:16.199001 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:16.198975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:20.440495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:20.440432    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:57:23.435380 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435335    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:57:23.435791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435715    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6860573f-a2ef-4318-afb4-8dee42c2c402" containerName="main"
Sep 10 17:57:23.435791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435729    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6860573f-a2ef-4318-afb4-8dee42c2c402" containerName="main"
Sep 10 17:57:23.435791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435747    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" containerName="main"
Sep 10 17:57:23.435791 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435753    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" containerName="main"
Sep 10 17:57:23.435935 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435817    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6860573f-a2ef-4318-afb4-8dee42c2c402" containerName="main"
Sep 10 17:57:23.435935 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.435828    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0921db9-b781-4af7-bfbf-eae08a9ea9ba" containerName="main"
Sep 10 17:57:23.439231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.439208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.441937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.441915    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"kube-root-ca.crt\""
Sep 10 17:57:23.442920 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.442894    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"default-dockercfg-qfnb6\""
Sep 10 17:57:23.443040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.442929    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"openshift-service-ca.crt\""
Sep 10 17:57:23.443040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.442994    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"sub-test-model-57474d-kserve-self-signed-certs\""
Sep 10 17:57:23.454481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.454427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:57:23.486044 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.486018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:23.567339 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567312    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.567442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smj4n\" (UniqueName: \"kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.567442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.567442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567420    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.567442 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567438    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.567622 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.567479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668497 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668623 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668507    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668623 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668583    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smj4n\" (UniqueName: \"kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668747 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668886 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668823    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.668886 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.668872    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.669085 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.669066    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.671074 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.671052    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.671309 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.671292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.677536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.677516    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smj4n\" (UniqueName: \"kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n\") pod \"sub-test-model-57474d-kserve-8697ddccc-q6cs6\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") " pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:23.750423 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:23.750367    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:24.023130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:24.023106    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:57:24.024900 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:57:24.024868    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd4606c2_01c2_477f_a7f9_21d9698829e7.slice/crio-5b368e46d346aca6d7f74d48d17776ae2edbf7c8add8a86815c26038ab54a71a WatchSource:0}: Error finding container 5b368e46d346aca6d7f74d48d17776ae2edbf7c8add8a86815c26038ab54a71a: Status 404 returned error can't find the container with id 5b368e46d346aca6d7f74d48d17776ae2edbf7c8add8a86815c26038ab54a71a
Sep 10 17:57:24.794373 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:24.794332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" event={"ID":"dd4606c2-01c2-477f-a7f9-21d9698829e7","Type":"ContainerStarted","Data":"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"}
Sep 10 17:57:24.794793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:24.794387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" event={"ID":"dd4606c2-01c2-477f-a7f9-21d9698829e7","Type":"ContainerStarted","Data":"5b368e46d346aca6d7f74d48d17776ae2edbf7c8add8a86815c26038ab54a71a"}
Sep 10 17:57:24.840210 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:24.840153    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" podStartSLOduration=1.840136228 podStartE2EDuration="1.840136228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:24.839611781 +0000 UTC m=+1260.357291943" watchObservedRunningTime="2026-09-10 17:57:24.840136228 +0000 UTC m=+1260.357816469"
Sep 10 17:57:26.725945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:26.725891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:26.958011 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:26.957967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:29.744160 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:29.744127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:57:33.750598 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:33.750557    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:33.751115 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:33.750641    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:33.763789 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:33.763759    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:33.854848 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:33.854823    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:57:43.034302 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:43.034257    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:57:47.921926 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:47.921882    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:53.741660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:53.741615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:53.913693 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:53.913666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:54.103306 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:54.103216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:54.429509 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:54.429481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:55.088524 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:55.088492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:56.660183 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.660148    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7699-dca86d"]
Sep 10 17:57:56.663484 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.663468    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:57:56.666147 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.666089    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w4zxj\""
Sep 10 17:57:56.668571 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.668545    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-dca86d"]
Sep 10 17:57:56.729947 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.729918    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n4p7\" (UniqueName: \"kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7\") pod \"test-curl-7699-dca86d\" (UID: \"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8\") " pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:57:56.831309 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.831283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n4p7\" (UniqueName: \"kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7\") pod \"test-curl-7699-dca86d\" (UID: \"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8\") " pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:57:56.840072 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.840049    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n4p7\" (UniqueName: \"kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7\") pod \"test-curl-7699-dca86d\" (UID: \"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8\") " pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:57:56.973983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:56.973907    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:57:57.248568 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.248383    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-dca86d"]
Sep 10 17:57:57.250854 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:57:57.250826    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d64cc19_b1cc_4cd8_b401_cffafa81ecf8.slice/crio-40e05924665eaa456772c6f113f077a6e06d8c694850fef7906c9de4d4ba53eb WatchSource:0}: Error finding container 40e05924665eaa456772c6f113f077a6e06d8c694850fef7906c9de4d4ba53eb: Status 404 returned error can't find the container with id 40e05924665eaa456772c6f113f077a6e06d8c694850fef7906c9de4d4ba53eb
Sep 10 17:57:57.289173 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.289149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:57.384020 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.383997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:57.978225 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.977197    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-dca86d" event={"ID":"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8","Type":"ContainerStarted","Data":"a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6"}
Sep 10 17:57:57.978225 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.977254    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-dca86d" event={"ID":"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8","Type":"ContainerStarted","Data":"40e05924665eaa456772c6f113f077a6e06d8c694850fef7906c9de4d4ba53eb"}
Sep 10 17:57:57.996244 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:57.996203    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7699-dca86d" podStartSLOduration=1.996189099 podStartE2EDuration="1.996189099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:57:57.995800191 +0000 UTC m=+1293.513480352" watchObservedRunningTime="2026-09-10 17:57:57.996189099 +0000 UTC m=+1293.513869260"
Sep 10 17:57:58.475536 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.475510    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:58.555573 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.555539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:58.617270 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.617245    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7699-dca86d"]
Sep 10 17:57:58.619739 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.619720    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7699-dca86d"]
Sep 10 17:57:58.637589 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.637566    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:58.736674 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.736620    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7699-03ac95"]
Sep 10 17:57:58.736951 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.736938    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" containerName="test-curl-7699-dca86d"
Sep 10 17:57:58.736999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.736952    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" containerName="test-curl-7699-dca86d"
Sep 10 17:57:58.737035 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.737026    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" containerName="test-curl-7699-dca86d"
Sep 10 17:57:58.740049 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.740035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:57:58.744511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.744488    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-03ac95"]
Sep 10 17:57:58.847788 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.847765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfgpd\" (UniqueName: \"kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd\") pod \"test-curl-7699-03ac95\" (UID: \"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681\") " pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:57:58.948926 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.948904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfgpd\" (UniqueName: \"kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd\") pod \"test-curl-7699-03ac95\" (UID: \"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681\") " pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:57:58.959396 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.959374    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfgpd\" (UniqueName: \"kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd\") pod \"test-curl-7699-03ac95\" (UID: \"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681\") " pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:57:58.982684 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:57:58.982663    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7699-dca86d" container="test-curl-7699-dca86d"
Sep 10 17:57:58.983683 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:58.983661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:59.050029 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.049976    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:57:59.101081 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.101045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:57:59.339713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.339645    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7699-03ac95"]
Sep 10 17:57:59.342860 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:57:59.342834    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6d76e33_99b3_40cb_87cf_e3ab6cd8d681.slice/crio-931862c862c11f3fab1fa0c04f8fee86a236d6a469ff76cbd6ffddaeed387ab3 WatchSource:0}: Error finding container 931862c862c11f3fab1fa0c04f8fee86a236d6a469ff76cbd6ffddaeed387ab3: Status 404 returned error can't find the container with id 931862c862c11f3fab1fa0c04f8fee86a236d6a469ff76cbd6ffddaeed387ab3
Sep 10 17:57:59.989531 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.989481    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-03ac95" event={"ID":"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681","Type":"ContainerStarted","Data":"ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c"}
Sep 10 17:57:59.989946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.989549    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7699-03ac95" event={"ID":"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681","Type":"ContainerStarted","Data":"931862c862c11f3fab1fa0c04f8fee86a236d6a469ff76cbd6ffddaeed387ab3"}
Sep 10 17:57:59.989946 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:57:59.989611    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7699-dca86d" podUID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" containerName="test-curl-7699-dca86d" containerID="cri-o://a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6" gracePeriod=2
Sep 10 17:58:00.644809 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:00.644760    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7699-03ac95" podStartSLOduration=2.644743698 podStartE2EDuration="2.644743698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:00.01098413 +0000 UTC m=+1295.528664292" watchObservedRunningTime="2026-09-10 17:58:00.644743698 +0000 UTC m=+1296.162423920"
Sep 10 17:58:00.645947 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:00.645928    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7699-03ac95"]
Sep 10 17:58:00.651274 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:00.651242    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7699-03ac95"]
Sep 10 17:58:00.994380 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:00.994350    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7699-03ac95" container="test-curl-7699-03ac95"
Sep 10 17:58:02.000120 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.000069    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7699-03ac95" podUID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" containerName="test-curl-7699-03ac95" containerID="cri-o://ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c" gracePeriod=2
Sep 10 17:58:02.137588 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.137567    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:58:02.278368 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.278302    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n4p7\" (UniqueName: \"kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7\") pod \"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8\" (UID: \"1d64cc19-b1cc-4cd8-b401-cffafa81ecf8\") "
Sep 10 17:58:02.280354 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.280318    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7" (OuterVolumeSpecName: "kube-api-access-9n4p7") pod "1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" (UID: "1d64cc19-b1cc-4cd8-b401-cffafa81ecf8"). InnerVolumeSpecName "kube-api-access-9n4p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:02.379081 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.379056    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n4p7\" (UniqueName: \"kubernetes.io/projected/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8-kube-api-access-9n4p7\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:02.858419 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.858386    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:58:02.880096 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:02.880070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:03.006017 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.005989    2571 generic.go:356] "Generic (PLEG): container finished" podID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" containerID="a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6" exitCode=137
Sep 10 17:58:03.006384 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.006090    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-dca86d"
Sep 10 17:58:03.008196 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.008172    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d64cc19-b1cc-4cd8-b401-cffafa81ecf8" path="/var/lib/kubelet/pods/1d64cc19-b1cc-4cd8-b401-cffafa81ecf8/volumes"
Sep 10 17:58:03.008685 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.008668    2571 scope.go:117] "RemoveContainer" containerID="a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6"
Sep 10 17:58:03.018231 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.018214    2571 scope.go:117] "RemoveContainer" containerID="a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6"
Sep 10 17:58:03.018681 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:03.018645    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6\": container with ID starting with a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6 not found: ID does not exist" containerID="a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6"
Sep 10 17:58:03.018790 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.018684    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6"} err="failed to get container status \"a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6\": rpc error: code = NotFound desc = could not find container \"a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6\": container with ID starting with a3576b30b7d4f9bb4f085d1f7c2d89684e27907ca1c5a6be7c8d8e12ddb5b4e6 not found: ID does not exist"
Sep 10 17:58:03.811626 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.811583    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:58:03.836913 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.836886    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:03.943447 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:03.943408    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:58:04.144220 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:04.144198    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:58:04.297086 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:04.297059    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfgpd\" (UniqueName: \"kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd\") pod \"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681\" (UID: \"e6d76e33-99b3-40cb-87cf-e3ab6cd8d681\") "
Sep 10 17:58:04.299357 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:04.299329    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd" (OuterVolumeSpecName: "kube-api-access-zfgpd") pod "e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" (UID: "e6d76e33-99b3-40cb-87cf-e3ab6cd8d681"). InnerVolumeSpecName "kube-api-access-zfgpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:04.398876 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:04.398841    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfgpd\" (UniqueName: \"kubernetes.io/projected/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681-kube-api-access-zfgpd\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:05.008646 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.008609    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" path="/var/lib/kubelet/pods/e6d76e33-99b3-40cb-87cf-e3ab6cd8d681/volumes"
Sep 10 17:58:05.019677 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.019647    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" containerID="ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c" exitCode=137
Sep 10 17:58:05.019804 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.019697    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7699-03ac95"
Sep 10 17:58:05.019804 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.019761    2571 scope.go:117] "RemoveContainer" containerID="ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c"
Sep 10 17:58:05.029271 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.029242    2571 scope.go:117] "RemoveContainer" containerID="ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c"
Sep 10 17:58:05.029580 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:05.029560    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c\": container with ID starting with ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c not found: ID does not exist" containerID="ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c"
Sep 10 17:58:05.029641 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:05.029588    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c"} err="failed to get container status \"ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c\": rpc error: code = NotFound desc = could not find container \"ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c\": container with ID starting with ce02cfdb42daee67aabd89e461fc879c660ea17e3317cc05ecd58d148c56161c not found: ID does not exist"
Sep 10 17:58:11.582912 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:11.582879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:11.653481 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:11.653434    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:17.878770 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:17.878728    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" podUID="a89f8e1d-1d61-4630-89e1-d530a363574f" containerName="main" containerID="cri-o://72811736e8309002381d6c70635b299b1f14aac59951b5ff9198f529e66528f5" gracePeriod=45
Sep 10 17:58:18.086181 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.086150    2571 generic.go:356] "Generic (PLEG): container finished" podID="a89f8e1d-1d61-4630-89e1-d530a363574f" containerID="72811736e8309002381d6c70635b299b1f14aac59951b5ff9198f529e66528f5" exitCode=0
Sep 10 17:58:18.086341 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.086199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" event={"ID":"a89f8e1d-1d61-4630-89e1-d530a363574f","Type":"ContainerDied","Data":"72811736e8309002381d6c70635b299b1f14aac59951b5ff9198f529e66528f5"}
Sep 10 17:58:18.121199 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.121179    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:58:18.206171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206102    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206142    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206171 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206170    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206425 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206194    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206425 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206289    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnvz4\" (UniqueName: \"kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206425 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache\") pod \"a89f8e1d-1d61-4630-89e1-d530a363574f\" (UID: \"a89f8e1d-1d61-4630-89e1-d530a363574f\") "
Sep 10 17:58:18.206612 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206447    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home" (OuterVolumeSpecName: "home") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:18.206612 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206525    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:18.206729 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206708    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.206729 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206704    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:18.206729 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.206722    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.208422 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.208400    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:18.208540 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.208492    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm" (OuterVolumeSpecName: "dshm") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:18.208540 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.208501    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4" (OuterVolumeSpecName: "kube-api-access-jnvz4") pod "a89f8e1d-1d61-4630-89e1-d530a363574f" (UID: "a89f8e1d-1d61-4630-89e1-d530a363574f"). InnerVolumeSpecName "kube-api-access-jnvz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:18.307268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.307245    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnvz4\" (UniqueName: \"kubernetes.io/projected/a89f8e1d-1d61-4630-89e1-d530a363574f-kube-api-access-jnvz4\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.307356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.307272    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.307356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.307287    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89f8e1d-1d61-4630-89e1-d530a363574f-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.307356 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.307295    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89f8e1d-1d61-4630-89e1-d530a363574f-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:18.831922 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:18.831862    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" podUID="dd4606c2-01c2-477f-a7f9-21d9698829e7" containerName="main" containerID="cri-o://af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95" gracePeriod=45
Sep 10 17:58:19.073235 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.073213    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:58:19.092861 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.092273    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"
Sep 10 17:58:19.092861 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.092272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8" event={"ID":"a89f8e1d-1d61-4630-89e1-d530a363574f","Type":"ContainerDied","Data":"193b9ad7532cd73d04f8d12080d9886578640010ee23cdb668d7150650925c14"}
Sep 10 17:58:19.092861 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.092491    2571 scope.go:117] "RemoveContainer" containerID="72811736e8309002381d6c70635b299b1f14aac59951b5ff9198f529e66528f5"
Sep 10 17:58:19.094819 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.094257    2571 generic.go:356] "Generic (PLEG): container finished" podID="dd4606c2-01c2-477f-a7f9-21d9698829e7" containerID="af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95" exitCode=0
Sep 10 17:58:19.095014 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.094425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" event={"ID":"dd4606c2-01c2-477f-a7f9-21d9698829e7","Type":"ContainerDied","Data":"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"}
Sep 10 17:58:19.095099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.095056    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6" event={"ID":"dd4606c2-01c2-477f-a7f9-21d9698829e7","Type":"ContainerDied","Data":"5b368e46d346aca6d7f74d48d17776ae2edbf7c8add8a86815c26038ab54a71a"}
Sep 10 17:58:19.095099 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.094797    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"
Sep 10 17:58:19.102920 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.102904    2571 scope.go:117] "RemoveContainer" containerID="af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"
Sep 10 17:58:19.110648 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.110625    2571 scope.go:117] "RemoveContainer" containerID="af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"
Sep 10 17:58:19.110947 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:19.110931    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95\": container with ID starting with af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95 not found: ID does not exist" containerID="af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"
Sep 10 17:58:19.111017 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.110956    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95"} err="failed to get container status \"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95\": rpc error: code = NotFound desc = could not find container \"af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95\": container with ID starting with af4d2f223c6b8fa093a5d68869f300a354c4b1506e839c6626900eeb9682af95 not found: ID does not exist"
Sep 10 17:58:19.112399 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.112379    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:58:19.115827 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.115805    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/sub-test-model-18ecb2-kserve-66b75889df-9w7l8"]
Sep 10 17:58:19.212995 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.212971    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213007    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213070    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213104 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213097    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smj4n\" (UniqueName: \"kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213262 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213127    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213262 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213164    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir\") pod \"dd4606c2-01c2-477f-a7f9-21d9698829e7\" (UID: \"dd4606c2-01c2-477f-a7f9-21d9698829e7\") "
Sep 10 17:58:19.213372 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213316    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home" (OuterVolumeSpecName: "home") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:19.213529 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213484    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:19.213615 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213544    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:19.213615 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.213570    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.215293 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.215267    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm" (OuterVolumeSpecName: "dshm") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:58:19.215407 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.215388    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:58:19.215407 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.215391    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n" (OuterVolumeSpecName: "kube-api-access-smj4n") pod "dd4606c2-01c2-477f-a7f9-21d9698829e7" (UID: "dd4606c2-01c2-477f-a7f9-21d9698829e7"). InnerVolumeSpecName "kube-api-access-smj4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:58:19.313911 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.313885    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.313911 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.313908    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smj4n\" (UniqueName: \"kubernetes.io/projected/dd4606c2-01c2-477f-a7f9-21d9698829e7-kube-api-access-smj4n\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.314041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.313918    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.314041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.313928    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd4606c2-01c2-477f-a7f9-21d9698829e7-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.314041 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.313935    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd4606c2-01c2-477f-a7f9-21d9698829e7-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:58:19.416688 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.416661    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:58:19.422629 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:19.422606    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/sub-test-model-57474d-kserve-8697ddccc-q6cs6"]
Sep 10 17:58:21.007764 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:21.007726    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89f8e1d-1d61-4630-89e1-d530a363574f" path="/var/lib/kubelet/pods/a89f8e1d-1d61-4630-89e1-d530a363574f/volumes"
Sep 10 17:58:21.008165 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:21.008097    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd4606c2-01c2-477f-a7f9-21d9698829e7" path="/var/lib/kubelet/pods/dd4606c2-01c2-477f-a7f9-21d9698829e7/volumes"
Sep 10 17:58:22.293037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:22.293007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:22.637071 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:22.637038    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:58:22.753152 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:22.753125    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:23.200018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:23.199985    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:23.336732 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:23.336698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:24.785933 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.785894    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786270    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89f8e1d-1d61-4630-89e1-d530a363574f" containerName="main"
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786285    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89f8e1d-1d61-4630-89e1-d530a363574f" containerName="main"
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786292    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" containerName="test-curl-7699-03ac95"
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786300    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" containerName="test-curl-7699-03ac95"
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786309    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd4606c2-01c2-477f-a7f9-21d9698829e7" containerName="main"
Sep 10 17:58:24.786348 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786315    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4606c2-01c2-477f-a7f9-21d9698829e7" containerName="main"
Sep 10 17:58:24.786567 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786372    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d76e33-99b3-40cb-87cf-e3ab6cd8d681" containerName="test-curl-7699-03ac95"
Sep 10 17:58:24.786567 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786381    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd4606c2-01c2-477f-a7f9-21d9698829e7" containerName="main"
Sep 10 17:58:24.786567 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.786388    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89f8e1d-1d61-4630-89e1-d530a363574f" containerName="main"
Sep 10 17:58:24.789405 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.789377    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.797237 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.797211    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"rate-test-model-18ecb2-kserve-self-signed-certs\""
Sep 10 17:58:24.798299 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.798276    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"openshift-service-ca.crt\""
Sep 10 17:58:24.798299 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.798297    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"kube-root-ca.crt\""
Sep 10 17:58:24.798502 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.798347    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"default-dockercfg-cfczq\""
Sep 10 17:58:24.824615 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.824580    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:58:24.932672 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.932633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:24.958249 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.958331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958253    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6pl6\" (UniqueName: \"kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.958331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.958331 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.958437 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:24.958437 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:24.958373    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059549 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059488    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059549 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6pl6\" (UniqueName: \"kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059556    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059586    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059634    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059718 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.059980 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.059959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.060062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.060002    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.060062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.060045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.061851 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.061830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.062509 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.062491    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"rate-test-model-18ecb2-kserve-self-signed-certs\""
Sep 10 17:58:25.068085 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.068067    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"kube-root-ca.crt\""
Sep 10 17:58:25.072587 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.072561    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.079207 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.079146    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"openshift-service-ca.crt\""
Sep 10 17:58:25.090078 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.090057    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6pl6\" (UniqueName: \"kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6\") pod \"rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") " pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.101130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.101109    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-18ecb2\"/\"default-dockercfg-cfczq\""
Sep 10 17:58:25.109144 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.109126    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:25.379018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:25.378993    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:58:25.381370 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:58:25.381344    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7abee267_7b65_4129_853b_37ba0af633a7.slice/crio-97f5fb67737b681d93e63c9d3bb03412e015bce560280677e095a2cd7b03e788 WatchSource:0}: Error finding container 97f5fb67737b681d93e63c9d3bb03412e015bce560280677e095a2cd7b03e788: Status 404 returned error can't find the container with id 97f5fb67737b681d93e63c9d3bb03412e015bce560280677e095a2cd7b03e788
Sep 10 17:58:26.138945 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:26.138899    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" event={"ID":"7abee267-7b65-4129-853b-37ba0af633a7","Type":"ContainerStarted","Data":"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"}
Sep 10 17:58:26.139412 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:26.138958    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" event={"ID":"7abee267-7b65-4129-853b-37ba0af633a7","Type":"ContainerStarted","Data":"97f5fb67737b681d93e63c9d3bb03412e015bce560280677e095a2cd7b03e788"}
Sep 10 17:58:26.159816 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:26.159765    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" podStartSLOduration=2.159748976 podStartE2EDuration="2.159748976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:26.158274391 +0000 UTC m=+1321.675954572" watchObservedRunningTime="2026-09-10 17:58:26.159748976 +0000 UTC m=+1321.677429191"
Sep 10 17:58:26.525800 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:26.525732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:26.619494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:26.619445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:27.639256 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.639224    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:58:27.643040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.643020    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.646081 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.646059    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-abad1a-serving-cert\""
Sep 10 17:58:27.658640 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.658601    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:58:27.694447 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.694422    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:27.780029 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.780004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.780133 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.780052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.780133 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.780086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.780133 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.780103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzj4\" (UniqueName: \"kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.793727 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.793704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:27.880965 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.880942    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.881077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.880969    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.881077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.880986    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzj4\" (UniqueName: \"kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.881077 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.881040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.881077 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:27.881042    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-serving-cert: secret "maas-api-e2e-api-b-abad1a-serving-cert" not found
Sep 10 17:58:27.881284 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:58:27.881122    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls podName:23136985-9b05-4100-a938-2d1d7dbe62c2 nodeName:}" failed. No retries permitted until 2026-09-10 17:58:28.381106992 +0000 UTC m=+1323.898787133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls") pod "maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" (UID: "23136985-9b05-4100-a938-2d1d7dbe62c2") : secret "maas-api-e2e-api-b-abad1a-serving-cert" not found
Sep 10 17:58:27.881692 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.881675    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.883554 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.883533    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:27.890497 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:27.890434    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzj4\" (UniqueName: \"kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:28.059851 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.059827    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:28.383769 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.383732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:28.384744 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.384719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:28.386268 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.386243    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") pod \"maas-api-e2e-api-b-abad1a-67d779f768-9w4bt\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:28.532574 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.532548    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:28.554507 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.554483    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:28.835817 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:28.835792    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:58:28.838244 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:58:28.838220    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23136985_9b05_4100_a938_2d1d7dbe62c2.slice/crio-631d88e687fdcb1a5a17b6281e7bdaab48612c1ac524f1c90307aa373346f9b7 WatchSource:0}: Error finding container 631d88e687fdcb1a5a17b6281e7bdaab48612c1ac524f1c90307aa373346f9b7: Status 404 returned error can't find the container with id 631d88e687fdcb1a5a17b6281e7bdaab48612c1ac524f1c90307aa373346f9b7
Sep 10 17:58:29.154494 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:29.154448    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" event={"ID":"23136985-9b05-4100-a938-2d1d7dbe62c2","Type":"ContainerStarted","Data":"631d88e687fdcb1a5a17b6281e7bdaab48612c1ac524f1c90307aa373346f9b7"}
Sep 10 17:58:30.160530 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:30.160498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" event={"ID":"23136985-9b05-4100-a938-2d1d7dbe62c2","Type":"ContainerStarted","Data":"ebb93442c1d740133cd244d12804cb44b89d746b6f14a8a9d93cbb1fbebd0642"}
Sep 10 17:58:30.160910 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:30.160635    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:30.248554 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:30.248496    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" podStartSLOduration=2.875755072 podStartE2EDuration="3.248474975s" totalImagesPullingTime="372.719903ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:30.247481945 +0000 UTC m=+1325.765162106" watchObservedRunningTime="2026-09-10 17:58:30.248474975 +0000 UTC m=+1325.766155130"
Sep 10 17:58:35.109317 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:35.109282    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:35.109317 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:35.109327    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:35.122179 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:35.122147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:35.193236 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:35.193193    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:58:36.170707 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:36.170678    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:58:38.140763 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:38.140729    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:58:45.104286 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:45.104248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:50.117722 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:50.117624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:51.489869 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:51.489833    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:54.565125 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:54.565087    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:55.032974 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.032939    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:58:55.036209 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.036188    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.038983 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.038951    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"kube-root-ca.crt\""
Sep 10 17:58:55.039941 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.039921    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"default-dockercfg-qfnb6\""
Sep 10 17:58:55.040094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.039927    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"openshift-service-ca.crt\""
Sep 10 17:58:55.040094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.040011    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-57474d\"/\"rate-test-model-57474d-kserve-self-signed-certs\""
Sep 10 17:58:55.051801 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.051772    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:58:55.070045 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.070020    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:58:55.097320 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.097413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097323    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.097413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097357    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.097413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097383    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.097538 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.097577 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.097542    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fd46\" (UniqueName: \"kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198559 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fd46\" (UniqueName: \"kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198602    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198674    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198793 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198714    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.198963 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.198942    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.199034 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.199000    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.199094 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.199032    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.200934 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.200915    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.201278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.201260    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.207219 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.207197    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fd46\" (UniqueName: \"kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46\") pod \"rate-test-model-57474d-kserve-579d59f98c-6xzwv\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") " pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.345495 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.345435    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:58:55.618168 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:55.618141    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:58:55.620649 ip-10-0-140-165 kubenswrapper[2571]: W0910 17:58:55.620620    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf6e9d95_9146_4837_a2e0_0709b1b4b6b2.slice/crio-ad28e437150646179249d29b557121858870a6277471173403927b2321883ca3 WatchSource:0}: Error finding container ad28e437150646179249d29b557121858870a6277471173403927b2321883ca3: Status 404 returned error can't find the container with id ad28e437150646179249d29b557121858870a6277471173403927b2321883ca3
Sep 10 17:58:56.284904 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:56.284867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" event={"ID":"df6e9d95-9146-4837-a2e0-0709b1b4b6b2","Type":"ContainerStarted","Data":"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"}
Sep 10 17:58:56.285061 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:56.284914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" event={"ID":"df6e9d95-9146-4837-a2e0-0709b1b4b6b2","Type":"ContainerStarted","Data":"ad28e437150646179249d29b557121858870a6277471173403927b2321883ca3"}
Sep 10 17:58:56.302941 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:56.302889    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" podStartSLOduration=1.3028739759999999 podStartE2EDuration="1.302873976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 17:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 17:58:56.301697748 +0000 UTC m=+1351.819377910" watchObservedRunningTime="2026-09-10 17:58:56.302873976 +0000 UTC m=+1351.820554138"
Sep 10 17:58:59.846928 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:58:59.846884    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:59:05.345769 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:05.345737    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:05.345769 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:05.345777    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:05.358037 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:05.358016    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:06.341048 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:06.341015    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:07.742559 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:07.742521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:09.934794 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:09.934758    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:10.035398 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:10.035368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:10.136644 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:10.136615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:10.233648 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:10.233584    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:10.627345 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:10.627307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:14.729578 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:14.729545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:17.022172 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:17.022131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:17.122578 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:17.122537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:17.252660 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:17.252623    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:59:17.323740 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:17.323668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:17.784471 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:17.784419    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:19.558387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:19.558341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:19.625949 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:19.625918    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:33.278229 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.278190    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:59:33.278758 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.278443    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" podUID="23136985-9b05-4100-a938-2d1d7dbe62c2" containerName="maas-api" containerID="cri-o://ebb93442c1d740133cd244d12804cb44b89d746b6f14a8a9d93cbb1fbebd0642" gracePeriod=30
Sep 10 17:59:33.312948 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.312921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:33.414022 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.413990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:33.469532 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.469501    2571 generic.go:356] "Generic (PLEG): container finished" podID="23136985-9b05-4100-a938-2d1d7dbe62c2" containerID="ebb93442c1d740133cd244d12804cb44b89d746b6f14a8a9d93cbb1fbebd0642" exitCode=0
Sep 10 17:59:33.469694 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.469582    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" event={"ID":"23136985-9b05-4100-a938-2d1d7dbe62c2","Type":"ContainerDied","Data":"ebb93442c1d740133cd244d12804cb44b89d746b6f14a8a9d93cbb1fbebd0642"}
Sep 10 17:59:33.512649 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.512623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:33.526166 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.526147    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:59:33.610629 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.610569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:33.715999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.715975    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") pod \"23136985-9b05-4100-a938-2d1d7dbe62c2\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") "
Sep 10 17:59:33.716119 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.716008    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dzj4\" (UniqueName: \"kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4\") pod \"23136985-9b05-4100-a938-2d1d7dbe62c2\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") "
Sep 10 17:59:33.716170 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.716139    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca\") pod \"23136985-9b05-4100-a938-2d1d7dbe62c2\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") "
Sep 10 17:59:33.716209 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.716191    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs\") pod \"23136985-9b05-4100-a938-2d1d7dbe62c2\" (UID: \"23136985-9b05-4100-a938-2d1d7dbe62c2\") "
Sep 10 17:59:33.716508 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.716480    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23136985-9b05-4100-a938-2d1d7dbe62c2" (UID: "23136985-9b05-4100-a938-2d1d7dbe62c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 17:59:33.718212 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.718184    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23136985-9b05-4100-a938-2d1d7dbe62c2" (UID: "23136985-9b05-4100-a938-2d1d7dbe62c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:33.718329 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.718240    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4" (OuterVolumeSpecName: "kube-api-access-2dzj4") pod "23136985-9b05-4100-a938-2d1d7dbe62c2" (UID: "23136985-9b05-4100-a938-2d1d7dbe62c2"). InnerVolumeSpecName "kube-api-access-2dzj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:33.718376 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.718356    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "23136985-9b05-4100-a938-2d1d7dbe62c2" (UID: "23136985-9b05-4100-a938-2d1d7dbe62c2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:33.817246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.817215    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23136985-9b05-4100-a938-2d1d7dbe62c2-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:33.817246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.817244    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:33.817246 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.817253    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23136985-9b05-4100-a938-2d1d7dbe62c2-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:33.817405 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.817261    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dzj4\" (UniqueName: \"kubernetes.io/projected/23136985-9b05-4100-a938-2d1d7dbe62c2-kube-api-access-2dzj4\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:33.870150 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:33.870127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:34.477315 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:34.477284    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"
Sep 10 17:59:34.477722 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:34.477320    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt" event={"ID":"23136985-9b05-4100-a938-2d1d7dbe62c2","Type":"ContainerDied","Data":"631d88e687fdcb1a5a17b6281e7bdaab48612c1ac524f1c90307aa373346f9b7"}
Sep 10 17:59:34.477722 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:34.477385    2571 scope.go:117] "RemoveContainer" containerID="ebb93442c1d740133cd244d12804cb44b89d746b6f14a8a9d93cbb1fbebd0642"
Sep 10 17:59:34.502764 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:34.502738    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:59:34.510770 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:34.510748    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-abad1a-67d779f768-9w4bt"]
Sep 10 17:59:35.008287 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:35.008255    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23136985-9b05-4100-a938-2d1d7dbe62c2" path="/var/lib/kubelet/pods/23136985-9b05-4100-a938-2d1d7dbe62c2/volumes"
Sep 10 17:59:36.442621 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:36.442587    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:37.425117 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:37.425077    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:59:39.981629 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:39.981591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:40.083130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:40.083090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:40.235724 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:40.235637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:40.924713 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:40.924681    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:41.634615 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:41.634585    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:59:41.658794 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:41.658760    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:42.563662 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:42.563630    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:59:42.618109 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:42.618081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:43.005407 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:43.005375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:43.316185 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:43.316104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:43.885582 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:43.885551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:46.917212 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:46.917175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:47.013733 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:47.013694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:48.181058 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:48.181030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:48.297647 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:48.297617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:48.652218 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:48.652191    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:48.758795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:48.758762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:48.850795 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:48.850770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:50.730890 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:50.730848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:51.489830 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:51.489793    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:52.194937 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:52.194902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:52.277992 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:52.277962    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:52.377888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:52.377861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:52.455413 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:52.455350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:52.877966 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:52.877939    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:55.839702 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:55.839665    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 17:59:56.653703 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.653669    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" podUID="7abee267-7b65-4129-853b-37ba0af633a7" containerName="main" containerID="cri-o://644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f" gracePeriod=45
Sep 10 17:59:56.894980 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.894956    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:59:56.987687 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987607    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.987687 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.987888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987697    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.987888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987722    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.987888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987770    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6pl6\" (UniqueName: \"kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.987888 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987787    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home\") pod \"7abee267-7b65-4129-853b-37ba0af633a7\" (UID: \"7abee267-7b65-4129-853b-37ba0af633a7\") "
Sep 10 17:59:56.988112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987945    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:56.988112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.987984    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:56.988112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.988088    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:56.988112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.988100    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home" (OuterVolumeSpecName: "home") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:56.988112 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.988109    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:56.989901 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.989867    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm" (OuterVolumeSpecName: "dshm") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:56.990003 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.989922    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:56.990003 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:56.989936    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6" (OuterVolumeSpecName: "kube-api-access-t6pl6") pod "7abee267-7b65-4129-853b-37ba0af633a7" (UID: "7abee267-7b65-4129-853b-37ba0af633a7"). InnerVolumeSpecName "kube-api-access-t6pl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:57.088871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.088841    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7abee267-7b65-4129-853b-37ba0af633a7-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.088871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.088865    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.088871 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.088874    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6pl6\" (UniqueName: \"kubernetes.io/projected/7abee267-7b65-4129-853b-37ba0af633a7-kube-api-access-t6pl6\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.089040 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.088883    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7abee267-7b65-4129-853b-37ba0af633a7-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.582863 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.582818    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" podUID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" containerName="main" containerID="cri-o://8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99" gracePeriod=45
Sep 10 17:59:57.586143 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.586110    2571 generic.go:356] "Generic (PLEG): container finished" podID="7abee267-7b65-4129-853b-37ba0af633a7" containerID="644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f" exitCode=0
Sep 10 17:59:57.586240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.586172    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"
Sep 10 17:59:57.586240 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.586190    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" event={"ID":"7abee267-7b65-4129-853b-37ba0af633a7","Type":"ContainerDied","Data":"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"}
Sep 10 17:59:57.586336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.586241    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt" event={"ID":"7abee267-7b65-4129-853b-37ba0af633a7","Type":"ContainerDied","Data":"97f5fb67737b681d93e63c9d3bb03412e015bce560280677e095a2cd7b03e788"}
Sep 10 17:59:57.586336 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.586262    2571 scope.go:117] "RemoveContainer" containerID="644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"
Sep 10 17:59:57.605876 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.605849    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:59:57.613130 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.613107    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-18ecb2/rate-test-model-18ecb2-kserve-5b49d9c75-sqbvt"]
Sep 10 17:59:57.638387 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.638369    2571 scope.go:117] "RemoveContainer" containerID="644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"
Sep 10 17:59:57.638713 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:59:57.638693    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f\": container with ID starting with 644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f not found: ID does not exist" containerID="644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"
Sep 10 17:59:57.638775 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.638722    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f"} err="failed to get container status \"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f\": rpc error: code = NotFound desc = could not find container \"644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f\": container with ID starting with 644fbb9a387e91834baa6a7f56047234c1fb9a160b2f9d81537c6e36a175235f not found: ID does not exist"
Sep 10 17:59:57.823988 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.823968    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:57.896069 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896046    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896076    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896099    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fd46\" (UniqueName: \"kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896119    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896188    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896216    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm\") pod \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\" (UID: \"df6e9d95-9146-4837-a2e0-0709b1b4b6b2\") "
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home" (OuterVolumeSpecName: "home") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896418    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:57.896560 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896434    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:57.897018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896792    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.897018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896825    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.897018 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.896840    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.898257 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.898227    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 17:59:57.898424 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.898395    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm" (OuterVolumeSpecName: "dshm") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 17:59:57.898511 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.898425    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46" (OuterVolumeSpecName: "kube-api-access-4fd46") pod "df6e9d95-9146-4837-a2e0-0709b1b4b6b2" (UID: "df6e9d95-9146-4837-a2e0-0709b1b4b6b2"). InnerVolumeSpecName "kube-api-access-4fd46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 17:59:57.997515 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.997480    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.997515 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.997508    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fd46\" (UniqueName: \"kubernetes.io/projected/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-kube-api-access-4fd46\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:57.997515 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:57.997521    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6e9d95-9146-4837-a2e0-0709b1b4b6b2-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 17:59:58.593999 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.593959    2571 generic.go:356] "Generic (PLEG): container finished" podID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" containerID="8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99" exitCode=0
Sep 10 17:59:58.594178 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.594016    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"
Sep 10 17:59:58.594178 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.594045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" event={"ID":"df6e9d95-9146-4837-a2e0-0709b1b4b6b2","Type":"ContainerDied","Data":"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"}
Sep 10 17:59:58.594178 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.594095    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv" event={"ID":"df6e9d95-9146-4837-a2e0-0709b1b4b6b2","Type":"ContainerDied","Data":"ad28e437150646179249d29b557121858870a6277471173403927b2321883ca3"}
Sep 10 17:59:58.594178 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.594116    2571 scope.go:117] "RemoveContainer" containerID="8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"
Sep 10 17:59:58.603390 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.603370    2571 scope.go:117] "RemoveContainer" containerID="8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"
Sep 10 17:59:58.603701 ip-10-0-140-165 kubenswrapper[2571]: E0910 17:59:58.603680    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99\": container with ID starting with 8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99 not found: ID does not exist" containerID="8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"
Sep 10 17:59:58.603759 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.603710    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99"} err="failed to get container status \"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99\": rpc error: code = NotFound desc = could not find container \"8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99\": container with ID starting with 8eff49cea8b9e0740d6a39efc345978df0a4003cd2c5dc07ec68cdfd9fb2aa99 not found: ID does not exist"
Sep 10 17:59:58.615174 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.615147    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:59:58.618836 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.618815    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-57474d/rate-test-model-57474d-kserve-579d59f98c-6xzwv"]
Sep 10 17:59:58.740401 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.740378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:58.799278 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:58.799258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:59.010062 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:59.010038    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7abee267-7b65-4129-853b-37ba0af633a7" path="/var/lib/kubelet/pods/7abee267-7b65-4129-853b-37ba0af633a7/volumes"
Sep 10 17:59:59.010502 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:59.010485    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" path="/var/lib/kubelet/pods/df6e9d95-9146-4837-a2e0-0709b1b4b6b2/volumes"
Sep 10 17:59:59.109784 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:59.109763    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 17:59:59.495881 ip-10-0-140-165 kubenswrapper[2571]: I0910 17:59:59.495853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:00.158102 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158068    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"]
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158435    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23136985-9b05-4100-a938-2d1d7dbe62c2" containerName="maas-api"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158448    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="23136985-9b05-4100-a938-2d1d7dbe62c2" containerName="maas-api"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158482    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" containerName="main"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158488    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" containerName="main"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158496    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abee267-7b65-4129-853b-37ba0af633a7" containerName="main"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158504    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abee267-7b65-4129-853b-37ba0af633a7" containerName="main"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158562    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7abee267-7b65-4129-853b-37ba0af633a7" containerName="main"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158569    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="23136985-9b05-4100-a938-2d1d7dbe62c2" containerName="maas-api"
Sep 10 18:00:00.158576 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.158574    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6e9d95-9146-4837-a2e0-0709b1b4b6b2" containerName="main"
Sep 10 18:00:00.162726 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.162707    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"]
Sep 10 18:00:00.162830 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.162817    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"
Sep 10 18:00:00.165541 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.165523    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k7zj6\""
Sep 10 18:00:00.165988 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.165973    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"
Sep 10 18:00:00.174050 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.174029    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"
Sep 10 18:00:00.182049 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.181985    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"]
Sep 10 18:00:00.184515 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.184484    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"
Sep 10 18:00:00.184994 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.184974    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"]
Sep 10 18:00:00.488983 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:00:00.488932    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95ed4df_66d3_4d0a_9bfb_05878218c4b1.slice/crio-f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7 WatchSource:0}: Error finding container f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7: Status 404 returned error can't find the container with id f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7
Sep 10 18:00:00.489847 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:00:00.489807    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236a0c4f_c076_46dd_81fb_1ec9eb9b5252.slice/crio-61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc WatchSource:0}: Error finding container 61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc: Status 404 returned error can't find the container with id 61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc
Sep 10 18:00:00.490855 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.490731    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"]
Sep 10 18:00:00.491301 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.491276    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 18:00:00.492697 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.492678    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"]
Sep 10 18:00:00.605851 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.605820    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f" event={"ID":"236a0c4f-c076-46dd-81fb-1ec9eb9b5252","Type":"ContainerStarted","Data":"61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc"}
Sep 10 18:00:00.607021 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:00.606983    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947" event={"ID":"e95ed4df-66d3-4d0a-9bfb-05878218c4b1","Type":"ContainerStarted","Data":"f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7"}
Sep 10 18:00:03.627780 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:03.627749    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f" event={"ID":"236a0c4f-c076-46dd-81fb-1ec9eb9b5252","Type":"ContainerStarted","Data":"43b37f36c5994c97dfc49c4559c841cc316035e90e7beab6b7a382eec2ba0793"}
Sep 10 18:00:03.629298 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:03.629275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947" event={"ID":"e95ed4df-66d3-4d0a-9bfb-05878218c4b1","Type":"ContainerStarted","Data":"570edecf3c81ff028f99c9ce26466573f8349af80021e916c5da6afcf2a34423"}
Sep 10 18:00:03.673429 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:03.673386    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f" podStartSLOduration=1.375537418 podStartE2EDuration="3.673370921s" totalImagesPullingTime="2.297833503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:03.671862049 +0000 UTC m=+1419.189542211" watchObservedRunningTime="2026-09-10 18:00:03.673370921 +0000 UTC m=+1419.191051116"
Sep 10 18:00:03.694447 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:03.694401    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947" podStartSLOduration=1.391742886 podStartE2EDuration="3.694384908s" totalImagesPullingTime="2.302642022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:03.692699689 +0000 UTC m=+1419.210379863" watchObservedRunningTime="2026-09-10 18:00:03.694384908 +0000 UTC m=+1419.212065067"
Sep 10 18:00:04.635629 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:04.635590    2571 generic.go:356] "Generic (PLEG): container finished" podID="236a0c4f-c076-46dd-81fb-1ec9eb9b5252" containerID="43b37f36c5994c97dfc49c4559c841cc316035e90e7beab6b7a382eec2ba0793" exitCode=0
Sep 10 18:00:04.636033 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:04.635679    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f" event={"ID":"236a0c4f-c076-46dd-81fb-1ec9eb9b5252","Type":"ContainerDied","Data":"43b37f36c5994c97dfc49c4559c841cc316035e90e7beab6b7a382eec2ba0793"}
Sep 10 18:00:04.637398 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:04.637367    2571 generic.go:356] "Generic (PLEG): container finished" podID="e95ed4df-66d3-4d0a-9bfb-05878218c4b1" containerID="570edecf3c81ff028f99c9ce26466573f8349af80021e916c5da6afcf2a34423" exitCode=0
Sep 10 18:00:04.637539 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:04.637440    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947" event={"ID":"e95ed4df-66d3-4d0a-9bfb-05878218c4b1","Type":"ContainerDied","Data":"570edecf3c81ff028f99c9ce26466573f8349af80021e916c5da6afcf2a34423"}
Sep 10 18:00:05.763310 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:05.763285    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"
Sep 10 18:00:05.802281 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:05.802262    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"
Sep 10 18:00:06.648575 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.648546    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"
Sep 10 18:00:06.648575 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.648556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f" event={"ID":"236a0c4f-c076-46dd-81fb-1ec9eb9b5252","Type":"ContainerDied","Data":"61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc"}
Sep 10 18:00:06.648823 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.648609    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61f1ec5d846e4f418e5a4dafc6ccaff0daee03893813c8e8752d8d320405c5bc"
Sep 10 18:00:06.650031 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.650012    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947"
Sep 10 18:00:06.650168 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.650039    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817720-df947" event={"ID":"e95ed4df-66d3-4d0a-9bfb-05878218c4b1","Type":"ContainerDied","Data":"f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7"}
Sep 10 18:00:06.650168 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:06.650069    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f749c544f6086deab19f45771fdd3bdaba95f2aec8c2b57aac3d73b39a2a95a7"
Sep 10 18:00:09.177012 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.176976    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:09.732655 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.732626    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 18:00:09.732901 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.732876    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" podUID="6678e15e-f73d-43e3-99f4-7623988e1c7e" containerName="maas-api" containerID="cri-o://22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9" gracePeriod=30
Sep 10 18:00:09.781938 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.781913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:09.882000 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.881972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:09.976748 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.976727    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 18:00:09.980832 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:09.980805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:10.051636 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.051576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:10.096244 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.096222    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs\") pod \"6678e15e-f73d-43e3-99f4-7623988e1c7e\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") "
Sep 10 18:00:10.096340 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.096259    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7j8n\" (UniqueName: \"kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n\") pod \"6678e15e-f73d-43e3-99f4-7623988e1c7e\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") "
Sep 10 18:00:10.096340 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.096334    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") pod \"6678e15e-f73d-43e3-99f4-7623988e1c7e\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") "
Sep 10 18:00:10.096422 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.096361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca\") pod \"6678e15e-f73d-43e3-99f4-7623988e1c7e\" (UID: \"6678e15e-f73d-43e3-99f4-7623988e1c7e\") "
Sep 10 18:00:10.096811 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.096781    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6678e15e-f73d-43e3-99f4-7623988e1c7e" (UID: "6678e15e-f73d-43e3-99f4-7623988e1c7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:00:10.098538 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.098514    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n" (OuterVolumeSpecName: "kube-api-access-z7j8n") pod "6678e15e-f73d-43e3-99f4-7623988e1c7e" (UID: "6678e15e-f73d-43e3-99f4-7623988e1c7e"). InnerVolumeSpecName "kube-api-access-z7j8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:00:10.098943 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.098923    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6678e15e-f73d-43e3-99f4-7623988e1c7e" (UID: "6678e15e-f73d-43e3-99f4-7623988e1c7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:00:10.099057 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.098940    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6678e15e-f73d-43e3-99f4-7623988e1c7e" (UID: "6678e15e-f73d-43e3-99f4-7623988e1c7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:00:10.196972 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.196948    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.196972 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.196970    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6678e15e-f73d-43e3-99f4-7623988e1c7e-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.197288 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.196980    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6678e15e-f73d-43e3-99f4-7623988e1c7e-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.197288 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.196989    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7j8n\" (UniqueName: \"kubernetes.io/projected/6678e15e-f73d-43e3-99f4-7623988e1c7e-kube-api-access-z7j8n\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:00:10.391035 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.391009    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:10.670977 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.670906    2571 generic.go:356] "Generic (PLEG): container finished" podID="6678e15e-f73d-43e3-99f4-7623988e1c7e" containerID="22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9" exitCode=0
Sep 10 18:00:10.670977 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.670961    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" event={"ID":"6678e15e-f73d-43e3-99f4-7623988e1c7e","Type":"ContainerDied","Data":"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"}
Sep 10 18:00:10.671129 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.671003    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx" event={"ID":"6678e15e-f73d-43e3-99f4-7623988e1c7e","Type":"ContainerDied","Data":"62a2424eaa669368c72531d7a064d1dc50029684aaa61eb628646c0318221261"}
Sep 10 18:00:10.671129 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.671027    2571 scope.go:117] "RemoveContainer" containerID="22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"
Sep 10 18:00:10.671129 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.670962    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"
Sep 10 18:00:10.681231 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.681215    2571 scope.go:117] "RemoveContainer" containerID="22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"
Sep 10 18:00:10.681560 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:00:10.681540    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9\": container with ID starting with 22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9 not found: ID does not exist" containerID="22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"
Sep 10 18:00:10.681628 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.681568    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9"} err="failed to get container status \"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9\": rpc error: code = NotFound desc = could not find container \"22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9\": container with ID starting with 22f21929d0cf78771380a0504175a51ab204f284a16bec31848893e3190ad1f9 not found: ID does not exist"
Sep 10 18:00:10.693160 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.693133    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 18:00:10.696739 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:10.696713    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-bdf47c-f8f9697cc-k5hxx"]
Sep 10 18:00:11.007860 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:11.007798    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6678e15e-f73d-43e3-99f4-7623988e1c7e" path="/var/lib/kubelet/pods/6678e15e-f73d-43e3-99f4-7623988e1c7e/volumes"
Sep 10 18:00:11.883163 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:11.883130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:12.527166 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:12.527133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:12.845654 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:12.845580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:12.946075 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:12.946034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:15.738471 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:15.738419    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:15.828759 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:15.828727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:16.640840 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.640805    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:00:16.862687 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.862652    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.862998    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95ed4df-66d3-4d0a-9bfb-05878218c4b1" containerName="cleanup"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863009    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95ed4df-66d3-4d0a-9bfb-05878218c4b1" containerName="cleanup"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863017    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="236a0c4f-c076-46dd-81fb-1ec9eb9b5252" containerName="cleanup"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863023    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="236a0c4f-c076-46dd-81fb-1ec9eb9b5252" containerName="cleanup"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863033    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6678e15e-f73d-43e3-99f4-7623988e1c7e" containerName="maas-api"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863039    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6678e15e-f73d-43e3-99f4-7623988e1c7e" containerName="maas-api"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863096    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="236a0c4f-c076-46dd-81fb-1ec9eb9b5252" containerName="cleanup"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863109    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6678e15e-f73d-43e3-99f4-7623988e1c7e" containerName="maas-api"
Sep 10 18:00:16.863157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.863119    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e95ed4df-66d3-4d0a-9bfb-05878218c4b1" containerName="cleanup"
Sep 10 18:00:16.867633 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.867611    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:16.870052 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.870036    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-db978e-serving-cert\""
Sep 10 18:00:16.878688 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.878669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:00:16.932854 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.932797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:16.951593 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.951571    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:16.951662 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.951614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:16.951662 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.951635    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:16.951662 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:16.951651    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg6zb\" (UniqueName: \"kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.035122 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.035099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:17.052479 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.052438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.052587 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.052502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.052587 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.052523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.052587 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.052539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6zb\" (UniqueName: \"kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.053231 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.053142    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.054942 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.054923    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.055011 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.054959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.066816 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.066798    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg6zb\" (UniqueName: \"kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb\") pod \"maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.179379 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.179359    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:17.219802 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.219768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:17.454344 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.454316    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:00:17.456441 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:00:17.456411    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a38104_b697_4b38_847b_8dda5c8f8fbf.slice/crio-a784276e654f6b582fc0fdcf07310ba97e5068a182a7139dc9d87760e8c329c2 WatchSource:0}: Error finding container a784276e654f6b582fc0fdcf07310ba97e5068a182a7139dc9d87760e8c329c2: Status 404 returned error can't find the container with id a784276e654f6b582fc0fdcf07310ba97e5068a182a7139dc9d87760e8c329c2
Sep 10 18:00:17.610167 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.610141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:17.708362 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.708284    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" event={"ID":"84a38104-b697-4b38-847b-8dda5c8f8fbf","Type":"ContainerStarted","Data":"a784276e654f6b582fc0fdcf07310ba97e5068a182a7139dc9d87760e8c329c2"}
Sep 10 18:00:17.709382 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.709346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:17.922157 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:17.922130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:18.715465 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:18.715414    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" event={"ID":"84a38104-b697-4b38-847b-8dda5c8f8fbf","Type":"ContainerStarted","Data":"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"}
Sep 10 18:00:18.715672 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:18.715536    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:18.732310 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:18.732035    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" podStartSLOduration=2.336444094 podStartE2EDuration="2.732016668s" totalImagesPullingTime="395.572574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:00:18.731832567 +0000 UTC m=+1434.249512733" watchObservedRunningTime="2026-09-10 18:00:18.732016668 +0000 UTC m=+1434.249696833"
Sep 10 18:00:24.725715 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:24.725688    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:00:28.234770 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:28.234730    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:00:34.083214 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:34.083183    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:00:40.729532 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:00:40.729500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:02.538582 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.538542    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:01:02.541262 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.541238    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.544863 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.544838    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-904bbd\"/\"kube-root-ca.crt\""
Sep 10 18:01:02.544979 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.544864    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-904bbd\"/\"openshift-service-ca.crt\""
Sep 10 18:01:02.544979 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.544864    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-904bbd\"/\"ipp-route-904bbd-kserve-self-signed-certs\""
Sep 10 18:01:02.544979 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.544842    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-904bbd\"/\"default-dockercfg-86cjp\""
Sep 10 18:01:02.550058 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.550033    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:01:02.618417 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618386    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.618539 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.618590 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618553    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.618637 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618592    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gccsj\" (UniqueName: \"kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.618695 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.618695 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.618681    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719422 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gccsj\" (UniqueName: \"kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719553 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719622 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719700 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719700 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719664    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719820 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719705    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719877 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719826    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.719928 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.719904    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.720049 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.720025    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.721963 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.721942    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.722218 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.722201    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.727903 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.727881    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gccsj\" (UniqueName: \"kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj\") pod \"ipp-route-904bbd-kserve-788cbdc7c9-zft8h\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") " pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:02.851783 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:02.851723    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:03.125811 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:03.125787    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:01:03.128043 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:01:03.128012    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeef18a7f_eeb9_4390_b8f8_2bd410e6f3bc.slice/crio-5ea465e24a36601b77003637ca8364f67d13a47f9cd3ece3c20a927629cf4fb1 WatchSource:0}: Error finding container 5ea465e24a36601b77003637ca8364f67d13a47f9cd3ece3c20a927629cf4fb1: Status 404 returned error can't find the container with id 5ea465e24a36601b77003637ca8364f67d13a47f9cd3ece3c20a927629cf4fb1
Sep 10 18:01:03.276768 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:03.276738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:03.927251 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:03.927218    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" event={"ID":"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc","Type":"ContainerStarted","Data":"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"}
Sep 10 18:01:03.927643 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:03.927263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" event={"ID":"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc","Type":"ContainerStarted","Data":"5ea465e24a36601b77003637ca8364f67d13a47f9cd3ece3c20a927629cf4fb1"}
Sep 10 18:01:03.945954 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:03.945903    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" podStartSLOduration=1.9458811520000001 podStartE2EDuration="1.945881152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:01:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:01:03.945192618 +0000 UTC m=+1479.462872780" watchObservedRunningTime="2026-09-10 18:01:03.945881152 +0000 UTC m=+1479.463561314"
Sep 10 18:01:07.036749 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:07.036711    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:01:12.852823 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:12.852765    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:12.853221 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:12.852851    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:12.865666 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:12.865637    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:12.983908 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:12.983879    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:01:20.307219 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:20.307153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:21.836575 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:21.836526    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:01:22.805712 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:22.805670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:29.318496 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:29.318464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:29.604149 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:29.604072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:30.324389 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:30.324327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:32.454754 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:32.454716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:32.551297 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:32.551258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:33.606505 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.606473    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:01:33.608857 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.608836    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.611731 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.611709    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-05a81f-serving-cert\""
Sep 10 18:01:33.623413 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.623393    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:01:33.642176 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.642122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:33.745386 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.745362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:33.769323 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.769304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.769397 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.769331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zxkt\" (UniqueName: \"kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.769444 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.769427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.769520 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.769488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.870733 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.870711    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.870822 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.870754    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.870822 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.870773    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zxkt\" (UniqueName: \"kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.870910 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.870826    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.870910 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:01:33.870862    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-serving-cert: secret "maas-api-e2e-ipp-cleanup-05a81f-serving-cert" not found
Sep 10 18:01:33.870992 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:01:33.870919    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls podName:53d3062a-40b1-4749-b10c-d15c38767090 nodeName:}" failed. No retries permitted until 2026-09-10 18:01:34.370904043 +0000 UTC m=+1509.888584186 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" (UID: "53d3062a-40b1-4749-b10c-d15c38767090") : secret "maas-api-e2e-ipp-cleanup-05a81f-serving-cert" not found
Sep 10 18:01:33.871405 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.871387    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.873345 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.873323    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:33.880143 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:33.880120    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zxkt\" (UniqueName: \"kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:34.024234 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.024206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:34.141159 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.141088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:34.246104 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.246071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:34.374939 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.374915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:34.377316 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.377295    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:34.519265 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.519208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:34.799310 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:34.799288    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:01:34.801956 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:01:34.801925    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d3062a_40b1_4749_b10c_d15c38767090.slice/crio-ccc33cd485aa644ed2c14909794e2ead0b77afac7a4c93f9ef396e3dc1196f4e WatchSource:0}: Error finding container ccc33cd485aa644ed2c14909794e2ead0b77afac7a4c93f9ef396e3dc1196f4e: Status 404 returned error can't find the container with id ccc33cd485aa644ed2c14909794e2ead0b77afac7a4c93f9ef396e3dc1196f4e
Sep 10 18:01:35.079261 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:35.079198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" event={"ID":"53d3062a-40b1-4749-b10c-d15c38767090","Type":"ContainerStarted","Data":"ccc33cd485aa644ed2c14909794e2ead0b77afac7a4c93f9ef396e3dc1196f4e"}
Sep 10 18:01:36.086047 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:36.086020    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" event={"ID":"53d3062a-40b1-4749-b10c-d15c38767090","Type":"ContainerStarted","Data":"d085a39a98f3482a4eca00a5c8566cc93102d866a7b3c418b689716a96c4c442"}
Sep 10 18:01:36.086423 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:36.086076    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:36.103156 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:36.103097    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" podStartSLOduration=2.746893839 podStartE2EDuration="3.103078337s" totalImagesPullingTime="356.184498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:01:36.101915811 +0000 UTC m=+1511.619595971" watchObservedRunningTime="2026-09-10 18:01:36.103078337 +0000 UTC m=+1511.620758499"
Sep 10 18:01:37.538892 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:37.538858    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:01:42.095780 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:42.095754    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:01:45.003298 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:45.003261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:56.508541 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:56.508463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:56.606756 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:56.606725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:01:57.493316 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:01:57.493292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:04.072979 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.072951    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:02:04.073427 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.073141    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" podUID="53d3062a-40b1-4749-b10c-d15c38767090" containerName="maas-api" containerID="cri-o://d085a39a98f3482a4eca00a5c8566cc93102d866a7b3c418b689716a96c4c442" gracePeriod=30
Sep 10 18:02:04.110395 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.110367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:04.211527 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.211435    2571 generic.go:356] "Generic (PLEG): container finished" podID="53d3062a-40b1-4749-b10c-d15c38767090" containerID="d085a39a98f3482a4eca00a5c8566cc93102d866a7b3c418b689716a96c4c442" exitCode=0
Sep 10 18:02:04.211527 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.211511    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" event={"ID":"53d3062a-40b1-4749-b10c-d15c38767090","Type":"ContainerDied","Data":"d085a39a98f3482a4eca00a5c8566cc93102d866a7b3c418b689716a96c4c442"}
Sep 10 18:02:04.211752 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.211559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:04.307331 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.307310    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:02:04.312483 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.312437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:04.402115 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402090    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs\") pod \"53d3062a-40b1-4749-b10c-d15c38767090\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") "
Sep 10 18:02:04.402329 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402145    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca\") pod \"53d3062a-40b1-4749-b10c-d15c38767090\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") "
Sep 10 18:02:04.402329 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402271    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") pod \"53d3062a-40b1-4749-b10c-d15c38767090\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") "
Sep 10 18:02:04.402329 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402317    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zxkt\" (UniqueName: \"kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt\") pod \"53d3062a-40b1-4749-b10c-d15c38767090\" (UID: \"53d3062a-40b1-4749-b10c-d15c38767090\") "
Sep 10 18:02:04.402542 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402511    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53d3062a-40b1-4749-b10c-d15c38767090" (UID: "53d3062a-40b1-4749-b10c-d15c38767090"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:02:04.402686 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.402669    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d3062a-40b1-4749-b10c-d15c38767090-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.404204 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.404183    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "53d3062a-40b1-4749-b10c-d15c38767090" (UID: "53d3062a-40b1-4749-b10c-d15c38767090"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:02:04.404329 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.404309    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53d3062a-40b1-4749-b10c-d15c38767090" (UID: "53d3062a-40b1-4749-b10c-d15c38767090"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:02:04.404413 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.404399    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt" (OuterVolumeSpecName: "kube-api-access-2zxkt") pod "53d3062a-40b1-4749-b10c-d15c38767090" (UID: "53d3062a-40b1-4749-b10c-d15c38767090"). InnerVolumeSpecName "kube-api-access-2zxkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:02:04.411442 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.411423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:04.503526 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.503509    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.503526 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.503526    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53d3062a-40b1-4749-b10c-d15c38767090-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.503659 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.503536    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zxkt\" (UniqueName: \"kubernetes.io/projected/53d3062a-40b1-4749-b10c-d15c38767090-kube-api-access-2zxkt\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:04.686424 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:04.686370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:05.217290 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:05.217265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d" event={"ID":"53d3062a-40b1-4749-b10c-d15c38767090","Type":"ContainerDied","Data":"ccc33cd485aa644ed2c14909794e2ead0b77afac7a4c93f9ef396e3dc1196f4e"}
Sep 10 18:02:05.217681 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:05.217274    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"
Sep 10 18:02:05.217681 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:05.217315    2571 scope.go:117] "RemoveContainer" containerID="d085a39a98f3482a4eca00a5c8566cc93102d866a7b3c418b689716a96c4c442"
Sep 10 18:02:05.233394 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:05.233373    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:02:05.237424 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:05.237405    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-05a81f-87d8f5c98-ss44d"]
Sep 10 18:02:07.007480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:07.007418    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d3062a-40b1-4749-b10c-d15c38767090" path="/var/lib/kubelet/pods/53d3062a-40b1-4749-b10c-d15c38767090/volumes"
Sep 10 18:02:08.362764 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:08.362738    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:02:10.944808 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:10.944779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:11.022394 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:11.022361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:11.227124 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:11.227049    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:14.979824 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:14.979789    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:28.033708 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:28.033674    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:02:30.475351 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:30.475318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:30.569689 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:30.569661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:30.673158 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:30.673126    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:30.744829 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:30.744763    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:31.035916 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:31.035842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:36.006363 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:36.006328    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:02:36.054435 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:36.054400    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:02:41.328371 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:41.328334    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:02:51.026045 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.025965    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" podUID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" containerName="main" containerID="cri-o://76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016" gracePeriod=45
Sep 10 18:02:51.280673 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.280614    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:02:51.392267 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392237    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392310    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392348    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392365    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gccsj\" (UniqueName: \"kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392397    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392480 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392446    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir\") pod \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\" (UID: \"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc\") "
Sep 10 18:02:51.392764 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392600    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:02:51.392764 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392753    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:51.392884 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392769    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home" (OuterVolumeSpecName: "home") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:02:51.392884 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.392786    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:02:51.394826 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.394793    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:02:51.394949 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.394861    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm" (OuterVolumeSpecName: "dshm") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:02:51.394949 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.394888    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj" (OuterVolumeSpecName: "kube-api-access-gccsj") pod "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" (UID: "eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc"). InnerVolumeSpecName "kube-api-access-gccsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:02:51.420741 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.420707    2571 generic.go:356] "Generic (PLEG): container finished" podID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" containerID="76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016" exitCode=0
Sep 10 18:02:51.420885 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.420747    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"
Sep 10 18:02:51.420885 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.420786    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" event={"ID":"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc","Type":"ContainerDied","Data":"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"}
Sep 10 18:02:51.420885 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.420826    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h" event={"ID":"eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc","Type":"ContainerDied","Data":"5ea465e24a36601b77003637ca8364f67d13a47f9cd3ece3c20a927629cf4fb1"}
Sep 10 18:02:51.420885 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.420845    2571 scope.go:117] "RemoveContainer" containerID="76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"
Sep 10 18:02:51.429564 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.429534    2571 scope.go:117] "RemoveContainer" containerID="76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"
Sep 10 18:02:51.429862 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:02:51.429844    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016\": container with ID starting with 76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016 not found: ID does not exist" containerID="76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"
Sep 10 18:02:51.429914 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.429871    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016"} err="failed to get container status \"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016\": rpc error: code = NotFound desc = could not find container \"76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016\": container with ID starting with 76915b7335af18ee10965d6c2128147c1fd8a553f9ecda79db4727a4019a6016 not found: ID does not exist"
Sep 10 18:02:51.442138 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.442107    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:02:51.446730 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.446706    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-904bbd/ipp-route-904bbd-kserve-788cbdc7c9-zft8h"]
Sep 10 18:02:51.493470 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.493434    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:51.493637 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.493482    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:51.493637 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.493492    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gccsj\" (UniqueName: \"kubernetes.io/projected/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-kube-api-access-gccsj\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:51.493637 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.493501    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:51.493637 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:51.493512    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:02:53.007306 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:02:53.007273    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" path="/var/lib/kubelet/pods/eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc/volumes"
Sep 10 18:03:12.902440 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:12.902403    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:13.349482 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:13.349363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:19.107534 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:19.107432    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:33.797951 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:33.797898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:33.870472 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:33.870431    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:34.090717 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:34.090633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:34.550209 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:34.550176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:44.887098 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:44.887061    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:03:44.887732 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:44.887312    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" podUID="84a38104-b697-4b38-847b-8dda5c8f8fbf" containerName="maas-api" containerID="cri-o://84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae" gracePeriod=30
Sep 10 18:03:44.928705 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:44.928668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:45.028042 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.027981    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:45.122921 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.122896    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:03:45.123575 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.123557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:45.169558 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.169440    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs\") pod \"84a38104-b697-4b38-847b-8dda5c8f8fbf\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") "
Sep 10 18:03:45.169558 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.169506    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls\") pod \"84a38104-b697-4b38-847b-8dda5c8f8fbf\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") "
Sep 10 18:03:45.169797 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.169581    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg6zb\" (UniqueName: \"kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb\") pod \"84a38104-b697-4b38-847b-8dda5c8f8fbf\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") "
Sep 10 18:03:45.169797 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.169694    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca\") pod \"84a38104-b697-4b38-847b-8dda5c8f8fbf\" (UID: \"84a38104-b697-4b38-847b-8dda5c8f8fbf\") "
Sep 10 18:03:45.170112 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.170078    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84a38104-b697-4b38-847b-8dda5c8f8fbf" (UID: "84a38104-b697-4b38-847b-8dda5c8f8fbf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:03:45.171835 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.171813    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84a38104-b697-4b38-847b-8dda5c8f8fbf" (UID: "84a38104-b697-4b38-847b-8dda5c8f8fbf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:03:45.171924 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.171908    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "84a38104-b697-4b38-847b-8dda5c8f8fbf" (UID: "84a38104-b697-4b38-847b-8dda5c8f8fbf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:03:45.171969 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.171921    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb" (OuterVolumeSpecName: "kube-api-access-sg6zb") pod "84a38104-b697-4b38-847b-8dda5c8f8fbf" (UID: "84a38104-b697-4b38-847b-8dda5c8f8fbf"). InnerVolumeSpecName "kube-api-access-sg6zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:03:45.192594 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.192547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:45.271352 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.271317    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg6zb\" (UniqueName: \"kubernetes.io/projected/84a38104-b697-4b38-847b-8dda5c8f8fbf-kube-api-access-sg6zb\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.271352 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.271345    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a38104-b697-4b38-847b-8dda5c8f8fbf-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.271352 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.271355    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.271352 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.271363    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84a38104-b697-4b38-847b-8dda5c8f8fbf-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:03:45.528361 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.528279    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:45.667146 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.667100    2571 generic.go:356] "Generic (PLEG): container finished" podID="84a38104-b697-4b38-847b-8dda5c8f8fbf" containerID="84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae" exitCode=0
Sep 10 18:03:45.667146 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.667136    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" event={"ID":"84a38104-b697-4b38-847b-8dda5c8f8fbf","Type":"ContainerDied","Data":"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"}
Sep 10 18:03:45.667412 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.667188    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"
Sep 10 18:03:45.667412 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.667227    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r" event={"ID":"84a38104-b697-4b38-847b-8dda5c8f8fbf","Type":"ContainerDied","Data":"a784276e654f6b582fc0fdcf07310ba97e5068a182a7139dc9d87760e8c329c2"}
Sep 10 18:03:45.667412 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.667251    2571 scope.go:117] "RemoveContainer" containerID="84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"
Sep 10 18:03:45.677010 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.676985    2571 scope.go:117] "RemoveContainer" containerID="84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"
Sep 10 18:03:45.677355 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:03:45.677326    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae\": container with ID starting with 84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae not found: ID does not exist" containerID="84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"
Sep 10 18:03:45.677469 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.677363    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae"} err="failed to get container status \"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae\": rpc error: code = NotFound desc = could not find container \"84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae\": container with ID starting with 84171aa2634795b411367039613aac24641304fdcc03d273092335f2963afbae not found: ID does not exist"
Sep 10 18:03:45.687476 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.687425    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:03:45.690734 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:45.690708    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-db978e-5f4446d99b-2z92r"]
Sep 10 18:03:46.066921 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:46.066884    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:47.009168 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:47.009133    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a38104-b697-4b38-847b-8dda5c8f8fbf" path="/var/lib/kubelet/pods/84a38104-b697-4b38-847b-8dda5c8f8fbf/volumes"
Sep 10 18:03:48.228069 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:48.228030    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:03:51.699892 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:51.699856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:51.801124 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:51.801088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:03:52.000794 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:03:52.000687    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:02.789219 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:02.789181    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"]
Sep 10 18:04:02.791850 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:02.791826    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-18ecb2-29817720-4zc5f"]
Sep 10 18:04:02.795255 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:02.795233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:02.897283 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:02.897255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:03.004321 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:03.004286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:03.007096 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:03.007073    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="236a0c4f-c076-46dd-81fb-1ec9eb9b5252" path="/var/lib/kubelet/pods/236a0c4f-c076-46dd-81fb-1ec9eb9b5252/volumes"
Sep 10 18:04:03.145857 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:03.145834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:03.435151 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:03.435090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:05.823301 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:05.823267    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:04:09.467961 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:09.467921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:09.545373 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:09.545346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:09.752616 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:09.752512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:10.176290 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:10.176259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:20.734002 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.733955    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 18:04:20.734512 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.734289    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" podUID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" containerName="maas-api" containerID="cri-o://b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e" gracePeriod=30
Sep 10 18:04:20.773886 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.773860    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:20.862646 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.862593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:20.961500 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.961470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:20.979759 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:20.979736    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 18:04:21.060966 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.060879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:21.146053 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.146033    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca\") pod \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") "
Sep 10 18:04:21.146195 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.146080    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d22d2\" (UniqueName: \"kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2\") pod \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") "
Sep 10 18:04:21.146195 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.146101    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs\") pod \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") "
Sep 10 18:04:21.146195 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.146138    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") pod \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\" (UID: \"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7\") "
Sep 10 18:04:21.146674 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.146371    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" (UID: "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:04:21.148368 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.148352    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" (UID: "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:04:21.148884 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.148863    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" (UID: "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:04:21.148952 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.148894    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2" (OuterVolumeSpecName: "kube-api-access-d22d2") pod "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" (UID: "4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7"). InnerVolumeSpecName "kube-api-access-d22d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:04:21.247445 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.247424    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:04:21.247445 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.247444    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:04:21.247584 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.247473    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:04:21.247584 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.247482    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d22d2\" (UniqueName: \"kubernetes.io/projected/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7-kube-api-access-d22d2\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:04:21.373391 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.373368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:21.820617 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.820546    2571 generic.go:356] "Generic (PLEG): container finished" podID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" containerID="b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e" exitCode=0
Sep 10 18:04:21.820617 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.820594    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" event={"ID":"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7","Type":"ContainerDied","Data":"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"}
Sep 10 18:04:21.821060 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.820631    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"
Sep 10 18:04:21.821060 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.820647    2571 scope.go:117] "RemoveContainer" containerID="b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"
Sep 10 18:04:21.821060 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.820636    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg" event={"ID":"4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7","Type":"ContainerDied","Data":"f1f5a1b5a2ba556f80f0386c6d77c1c95c97f26c868e25b904c10828b8605b52"}
Sep 10 18:04:21.830159 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.830141    2571 scope.go:117] "RemoveContainer" containerID="b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"
Sep 10 18:04:21.830442 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:04:21.830412    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e\": container with ID starting with b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e not found: ID does not exist" containerID="b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"
Sep 10 18:04:21.830526 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.830467    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e"} err="failed to get container status \"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e\": rpc error: code = NotFound desc = could not find container \"b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e\": container with ID starting with b679ef08d2b507324e5fd59e86bb8ed92185810c35eef504f3a044eb3a2f3f2e not found: ID does not exist"
Sep 10 18:04:21.842196 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.842173    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 18:04:21.848595 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:21.848568    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-57474d-74859bf8cf-rh5dg"]
Sep 10 18:04:23.006427 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:23.006394    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" path="/var/lib/kubelet/pods/4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7/volumes"
Sep 10 18:04:23.337279 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:23.337202    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:04:27.632895 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:27.632856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:27.760272 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:27.760242    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:27.961027 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:27.960956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:04:32.432062 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:04:32.432025    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:05:08.886669 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:08.886579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:05:08.986860 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:08.986831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:05:51.969819 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:51.969777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:05:52.842128 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:52.842091    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:05:54.000790 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:54.000758    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:05:55.592152 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:05:55.592111    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:06:17.823106 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:17.823013    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:06:25.171053 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:25.171018    2571 scope.go:117] "RemoveContainer" containerID="43b37f36c5994c97dfc49c4559c841cc316035e90e7beab6b7a382eec2ba0793"
Sep 10 18:06:25.423259 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:25.423191    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:06:32.329161 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:32.329126    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:06:37.429093 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:37.429056    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:06:51.032063 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:51.032024    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:06:59.030891 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:06:59.030858    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:08:01.017838 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:01.017753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:08:02.158028 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:02.157987    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:08:28.130510 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:28.130477    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:08:34.721653 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:34.721615    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:08:41.725381 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:41.725342    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:08:41.851292 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:41.851258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:08:48.635720 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:48.635686    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:08:51.745310 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745269    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745632    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a38104-b697-4b38-847b-8dda5c8f8fbf" containerName="maas-api"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745645    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a38104-b697-4b38-847b-8dda5c8f8fbf" containerName="maas-api"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745662    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" containerName="main"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745668    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" containerName="main"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745679    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d3062a-40b1-4749-b10c-d15c38767090" containerName="maas-api"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745687    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d3062a-40b1-4749-b10c-d15c38767090" containerName="maas-api"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745701    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" containerName="maas-api"
Sep 10 18:08:51.745713 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745706    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" containerName="maas-api"
Sep 10 18:08:51.745965 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745761    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef18a7f-eeb9-4390-b8f8-2bd410e6f3bc" containerName="main"
Sep 10 18:08:51.745965 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745770    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8ab0a4-3d8b-430b-8924-a5f8dfaec5a7" containerName="maas-api"
Sep 10 18:08:51.745965 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745777    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a38104-b697-4b38-847b-8dda5c8f8fbf" containerName="maas-api"
Sep 10 18:08:51.745965 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.745782    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d3062a-40b1-4749-b10c-d15c38767090" containerName="maas-api"
Sep 10 18:08:51.748726 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.748706    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.751426 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.751406    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d37e65ae-kserve-self-signed-certs\""
Sep 10 18:08:51.758101 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758076    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.758223 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758106    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.758223 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758126    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.758223 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758149    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.758223 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758215    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.758389 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.758256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6qh\" (UniqueName: \"kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.765298 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.765274    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:08:51.788912 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.788877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:08:51.859314 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6qh\" (UniqueName: \"kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859532 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859532 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859532 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859397    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859532 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859707 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859530    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859795 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859864 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.859864 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.859857    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.861906 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.861880    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.862101 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.862085    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:51.869432 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:51.869406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6qh\" (UniqueName: \"kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh\") pod \"e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") " pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:52.057386 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:52.057300    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:08:52.331593 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:52.331566    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:08:52.333994 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:08:52.333967    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd29bf897_7c6a_443b_a532_ac06c3a36991.slice/crio-174cb648de15e96c193d8e18c418258003ee7a858da68b68dd0eb9e2ecfe4a20 WatchSource:0}: Error finding container 174cb648de15e96c193d8e18c418258003ee7a858da68b68dd0eb9e2ecfe4a20: Status 404 returned error can't find the container with id 174cb648de15e96c193d8e18c418258003ee7a858da68b68dd0eb9e2ecfe4a20
Sep 10 18:08:53.025764 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:53.025728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" event={"ID":"d29bf897-7c6a-443b-a532-ac06c3a36991","Type":"ContainerStarted","Data":"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"}
Sep 10 18:08:53.026175 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:53.025773    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" event={"ID":"d29bf897-7c6a-443b-a532-ac06c3a36991","Type":"ContainerStarted","Data":"174cb648de15e96c193d8e18c418258003ee7a858da68b68dd0eb9e2ecfe4a20"}
Sep 10 18:08:53.041681 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:53.041635    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" podStartSLOduration=2.041621947 podStartE2EDuration="2.041621947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:08:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:08:53.040707855 +0000 UTC m=+1948.558388017" watchObservedRunningTime="2026-09-10 18:08:53.041621947 +0000 UTC m=+1948.559302108"
Sep 10 18:08:54.325725 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:08:54.325686    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:02.057752 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:02.057708    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:02.058251 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:02.057799    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:02.072010 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:02.071986    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:03.087426 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:03.087396    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:11.107798 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:11.107756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:09:12.111232 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:12.111193    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:09:12.132273 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:12.132240    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:09:14.219430 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:14.219389    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:22.627442 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:22.627361    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:27.131249 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.131186    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" podUID="d29bf897-7c6a-443b-a532-ac06c3a36991" containerName="main" containerID="cri-o://cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4" gracePeriod=45
Sep 10 18:09:27.376749 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.376728    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:27.443327 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443269    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443327 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443301    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443544 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443330    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443544 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443380    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443544 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443419    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt6qh\" (UniqueName: \"kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443544 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443494    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm\") pod \"d29bf897-7c6a-443b-a532-ac06c3a36991\" (UID: \"d29bf897-7c6a-443b-a532-ac06c3a36991\") "
Sep 10 18:09:27.443747 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443573    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache" (OuterVolumeSpecName: "model-cache") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:09:27.443747 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443595    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:09:27.443747 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443640    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home" (OuterVolumeSpecName: "home") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:09:27.443887 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443775    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-model-cache\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:27.443887 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443792    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-tmp-dir\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:27.443887 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.443804    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-home\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:27.445508 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.445483    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:09:27.445615 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.445590    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm" (OuterVolumeSpecName: "dshm") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 18:09:27.445678 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.445647    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh" (OuterVolumeSpecName: "kube-api-access-vt6qh") pod "d29bf897-7c6a-443b-a532-ac06c3a36991" (UID: "d29bf897-7c6a-443b-a532-ac06c3a36991"). InnerVolumeSpecName "kube-api-access-vt6qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:09:27.544589 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.544551    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt6qh\" (UniqueName: \"kubernetes.io/projected/d29bf897-7c6a-443b-a532-ac06c3a36991-kube-api-access-vt6qh\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:27.544589 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.544590    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d29bf897-7c6a-443b-a532-ac06c3a36991-dshm\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:27.544752 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:27.544619    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d29bf897-7c6a-443b-a532-ac06c3a36991-tls-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:28.188997 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.188960    2571 generic.go:356] "Generic (PLEG): container finished" podID="d29bf897-7c6a-443b-a532-ac06c3a36991" containerID="cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4" exitCode=0
Sep 10 18:09:28.189488 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.189015    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"
Sep 10 18:09:28.189488 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.189050    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" event={"ID":"d29bf897-7c6a-443b-a532-ac06c3a36991","Type":"ContainerDied","Data":"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"}
Sep 10 18:09:28.189488 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.189093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w" event={"ID":"d29bf897-7c6a-443b-a532-ac06c3a36991","Type":"ContainerDied","Data":"174cb648de15e96c193d8e18c418258003ee7a858da68b68dd0eb9e2ecfe4a20"}
Sep 10 18:09:28.189488 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.189112    2571 scope.go:117] "RemoveContainer" containerID="cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"
Sep 10 18:09:28.198245 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.198230    2571 scope.go:117] "RemoveContainer" containerID="cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"
Sep 10 18:09:28.198523 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:09:28.198499    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4\": container with ID starting with cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4 not found: ID does not exist" containerID="cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"
Sep 10 18:09:28.198622 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.198530    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4"} err="failed to get container status \"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4\": rpc error: code = NotFound desc = could not find container \"cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4\": container with ID starting with cb2f565ed43f02481aa92174117083b271c3c9607b229f46fe1ab04dcde6c7e4 not found: ID does not exist"
Sep 10 18:09:28.209519 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.209499    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:09:28.212757 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:28.212737    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d37e65ae-kserve-fcf8c4bc5-4bw7w"]
Sep 10 18:09:29.006887 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:29.006850    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d29bf897-7c6a-443b-a532-ac06c3a36991" path="/var/lib/kubelet/pods/d29bf897-7c6a-443b-a532-ac06c3a36991/volumes"
Sep 10 18:09:30.052699 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.052667    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:30.053056 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.052983    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d29bf897-7c6a-443b-a532-ac06c3a36991" containerName="main"
Sep 10 18:09:30.053056 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.052995    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d29bf897-7c6a-443b-a532-ac06c3a36991" containerName="main"
Sep 10 18:09:30.053056 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.053052    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d29bf897-7c6a-443b-a532-ac06c3a36991" containerName="main"
Sep 10 18:09:30.057325 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.057308    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.060806 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.060782    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 18:09:30.060806 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.060799    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 18:09:30.060806 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.060787    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 18:09:30.060806 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.060813    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 18:09:30.061081 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.060804    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tdclw\""
Sep 10 18:09:30.066773 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.066752    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:30.164694 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.164656    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.164832 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.164716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkmt2\" (UniqueName: \"kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.164832 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.164788    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.164832 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.164819    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.265606 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.265575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.265712 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.265611    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.265836 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.265815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.265906 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.265887    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkmt2\" (UniqueName: \"kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.266286 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.266257    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.268338 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.268313    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.268431 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.268381    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.273249 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.273229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkmt2\" (UniqueName: \"kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2\") pod \"maas-api-6ff67d7f48-s27hv\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") " pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.369088 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.369069    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:30.521440 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.521398    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:30.638072 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.638014    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:30.640569 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:09:30.640536    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde438dc_0636_4103_8a68_27643d5fa5df.slice/crio-0e3f1d2b081a4ccae49046d72513cf3dcc1ef5a936bbed09d53436a634f71450 WatchSource:0}: Error finding container 0e3f1d2b081a4ccae49046d72513cf3dcc1ef5a936bbed09d53436a634f71450: Status 404 returned error can't find the container with id 0e3f1d2b081a4ccae49046d72513cf3dcc1ef5a936bbed09d53436a634f71450
Sep 10 18:09:30.641925 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:30.641905    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 18:09:31.205754 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:31.205724    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" event={"ID":"dde438dc-0636-4103-8a68-27643d5fa5df","Type":"ContainerStarted","Data":"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"}
Sep 10 18:09:31.206096 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:31.205771    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" event={"ID":"dde438dc-0636-4103-8a68-27643d5fa5df","Type":"ContainerStarted","Data":"0e3f1d2b081a4ccae49046d72513cf3dcc1ef5a936bbed09d53436a634f71450"}
Sep 10 18:09:31.206096 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:31.205824    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:31.224366 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:31.224325    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" podStartSLOduration=0.869611098 podStartE2EDuration="1.224311069s" totalImagesPullingTime="354.699971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:09:31.223215022 +0000 UTC m=+1986.740895184" watchObservedRunningTime="2026-09-10 18:09:31.224311069 +0000 UTC m=+1986.741991231"
Sep 10 18:09:37.215643 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:37.215609    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:37.322643 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:37.322613    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:37.971957 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:37.971919    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"]
Sep 10 18:09:37.975578 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:37.975553    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:37.985286 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:37.985254    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"]
Sep 10 18:09:38.127067 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.127036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-metrics-certs\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.127216 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.127074    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de0056-d57a-4de7-98ee-5b359dba9cec-openshift-service-ca\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.127216 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.127093    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-maas-api-tls\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.127216 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.127117    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bx59\" (UniqueName: \"kubernetes.io/projected/d8de0056-d57a-4de7-98ee-5b359dba9cec-kube-api-access-4bx59\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.228429 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.228353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-metrics-certs\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.228429 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.228416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de0056-d57a-4de7-98ee-5b359dba9cec-openshift-service-ca\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.228801 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.228446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-maas-api-tls\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.228801 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.228495    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bx59\" (UniqueName: \"kubernetes.io/projected/d8de0056-d57a-4de7-98ee-5b359dba9cec-kube-api-access-4bx59\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.229204 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.229173    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8de0056-d57a-4de7-98ee-5b359dba9cec-openshift-service-ca\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.231116 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.231088    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-metrics-certs\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.231206 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.231125    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8de0056-d57a-4de7-98ee-5b359dba9cec-maas-api-tls\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.236650 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.236629    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bx59\" (UniqueName: \"kubernetes.io/projected/d8de0056-d57a-4de7-98ee-5b359dba9cec-kube-api-access-4bx59\") pod \"maas-api-84b747cc96-jghtt\" (UID: \"d8de0056-d57a-4de7-98ee-5b359dba9cec\") " pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.288903 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.288879    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:38.561698 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:38.561669    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"]
Sep 10 18:09:38.563697 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:09:38.563663    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8de0056_d57a_4de7_98ee_5b359dba9cec.slice/crio-51197fa78c063ee1dc66f7fcba47c8d10165e463a1a7b1da668d04383c7a8c68 WatchSource:0}: Error finding container 51197fa78c063ee1dc66f7fcba47c8d10165e463a1a7b1da668d04383c7a8c68: Status 404 returned error can't find the container with id 51197fa78c063ee1dc66f7fcba47c8d10165e463a1a7b1da668d04383c7a8c68
Sep 10 18:09:39.245296 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:39.245215    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt" event={"ID":"d8de0056-d57a-4de7-98ee-5b359dba9cec","Type":"ContainerStarted","Data":"6ea171696bf52c0a96001d6b327ad501f159f13010b81eff2bd78860b6a4df63"}
Sep 10 18:09:39.245296 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:39.245265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt" event={"ID":"d8de0056-d57a-4de7-98ee-5b359dba9cec","Type":"ContainerStarted","Data":"51197fa78c063ee1dc66f7fcba47c8d10165e463a1a7b1da668d04383c7a8c68"}
Sep 10 18:09:39.245732 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:39.245322    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:39.261609 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:39.261563    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt" podStartSLOduration=1.908615951 podStartE2EDuration="2.261549032s" totalImagesPullingTime="352.933081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 18:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:09:39.260295624 +0000 UTC m=+1994.777975786" watchObservedRunningTime="2026-09-10 18:09:39.261549032 +0000 UTC m=+1994.779229193"
Sep 10 18:09:44.125924 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:44.125882    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:45.255832 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:45.255798    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84b747cc96-jghtt"
Sep 10 18:09:45.301344 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:45.301310    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:45.301545 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:45.301493    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" podUID="dde438dc-0636-4103-8a68-27643d5fa5df" containerName="maas-api" containerID="cri-o://e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00" gracePeriod=30
Sep 10 18:09:46.533555 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.533533    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:46.597711 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.597677    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls\") pod \"dde438dc-0636-4103-8a68-27643d5fa5df\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") "
Sep 10 18:09:46.597867 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.597768    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkmt2\" (UniqueName: \"kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2\") pod \"dde438dc-0636-4103-8a68-27643d5fa5df\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") "
Sep 10 18:09:46.597867 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.597804    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs\") pod \"dde438dc-0636-4103-8a68-27643d5fa5df\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") "
Sep 10 18:09:46.597867 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.597845    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca\") pod \"dde438dc-0636-4103-8a68-27643d5fa5df\" (UID: \"dde438dc-0636-4103-8a68-27643d5fa5df\") "
Sep 10 18:09:46.598235 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.598209    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dde438dc-0636-4103-8a68-27643d5fa5df" (UID: "dde438dc-0636-4103-8a68-27643d5fa5df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 18:09:46.599912 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.599883    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dde438dc-0636-4103-8a68-27643d5fa5df" (UID: "dde438dc-0636-4103-8a68-27643d5fa5df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:09:46.600024 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.599920    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dde438dc-0636-4103-8a68-27643d5fa5df" (UID: "dde438dc-0636-4103-8a68-27643d5fa5df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 18:09:46.600024 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.599935    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2" (OuterVolumeSpecName: "kube-api-access-hkmt2") pod "dde438dc-0636-4103-8a68-27643d5fa5df" (UID: "dde438dc-0636-4103-8a68-27643d5fa5df"). InnerVolumeSpecName "kube-api-access-hkmt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 18:09:46.698868 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.698798    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkmt2\" (UniqueName: \"kubernetes.io/projected/dde438dc-0636-4103-8a68-27643d5fa5df-kube-api-access-hkmt2\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:46.698868 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.698824    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-metrics-certs\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:46.698868 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.698837    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dde438dc-0636-4103-8a68-27643d5fa5df-openshift-service-ca\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:46.698868 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:46.698849    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dde438dc-0636-4103-8a68-27643d5fa5df-maas-api-tls\") on node \"ip-10-0-140-165.ec2.internal\" DevicePath \"\""
Sep 10 18:09:47.288641 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.288565    2571 generic.go:356] "Generic (PLEG): container finished" podID="dde438dc-0636-4103-8a68-27643d5fa5df" containerID="e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00" exitCode=0
Sep 10 18:09:47.288641 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.288628    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"
Sep 10 18:09:47.288815 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.288650    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" event={"ID":"dde438dc-0636-4103-8a68-27643d5fa5df","Type":"ContainerDied","Data":"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"}
Sep 10 18:09:47.288815 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.288704    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv" event={"ID":"dde438dc-0636-4103-8a68-27643d5fa5df","Type":"ContainerDied","Data":"0e3f1d2b081a4ccae49046d72513cf3dcc1ef5a936bbed09d53436a634f71450"}
Sep 10 18:09:47.288815 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.288727    2571 scope.go:117] "RemoveContainer" containerID="e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"
Sep 10 18:09:47.296843 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.296825    2571 scope.go:117] "RemoveContainer" containerID="e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"
Sep 10 18:09:47.297119 ip-10-0-140-165 kubenswrapper[2571]: E0910 18:09:47.297103    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00\": container with ID starting with e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00 not found: ID does not exist" containerID="e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"
Sep 10 18:09:47.297197 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.297129    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00"} err="failed to get container status \"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00\": rpc error: code = NotFound desc = could not find container \"e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00\": container with ID starting with e239e336303655457150dd24b316b8dae2f9954277f83ab1ff1ee04766511c00 not found: ID does not exist"
Sep 10 18:09:47.306305 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.306274    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:47.311385 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:47.311365    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ff67d7f48-s27hv"]
Sep 10 18:09:49.007028 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:49.006993    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde438dc-0636-4103-8a68-27643d5fa5df" path="/var/lib/kubelet/pods/dde438dc-0636-4103-8a68-27643d5fa5df/volumes"
Sep 10 18:09:49.929619 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:49.929581    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:56.829925 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:56.829887    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7g2kt"]
Sep 10 18:09:59.785367 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:09:59.785331    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-856bfbb5f6-r8jtd_147919b5-2162-4add-9224-eb9f5f0ddf3c/authorino/0.log"
Sep 10 18:10:05.676413 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:05.676380    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-m897n_c6b2dca6-2edb-4cc1-a39a-22426bf01dd4/copy-manifests/0.log"
Sep 10 18:10:05.683041 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:05.683004    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-m897n_c6b2dca6-2edb-4cc1-a39a-22426bf01dd4/manager/0.log"
Sep 10 18:10:06.358520 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:06.358478    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-84b747cc96-jghtt_d8de0056-d57a-4de7-98ee-5b359dba9cec/maas-api/0.log"
Sep 10 18:10:06.688338 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:06.688312    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817720-df947_e95ed4df-66d3-4d0a-9bfb-05878218c4b1/cleanup/0.log"
Sep 10 18:10:07.557900 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:07.557868    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-856bfbb5f6-r8jtd_147919b5-2162-4add-9224-eb9f5f0ddf3c/authorino/0.log"
Sep 10 18:10:07.886917 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:07.886890    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-4z92c_8e2e0e76-065a-46cd-b9e5-768fa5263b10/kuadrant-console-plugin/0.log"
Sep 10 18:10:08.122536 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:08.122518    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-7g2kt_f608ec2a-3953-46e1-a116-79b27d453365/limitador/0.log"
Sep 10 18:10:08.256368 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:08.256300    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-nvcx5_cfce37ba-84e0-45b2-a10b-c7512c1cf106/manager/0.log"
Sep 10 18:10:08.858840 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:08.858808    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-bstbn_807feed5-8bc2-4b14-82ab-0bfa85face7b/kube-auth-proxy/0.log"
Sep 10 18:10:10.005407 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:10.005376    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-mk97m_1716aa43-1264-49b5-9024-35f0dc96f739/main/0.log"
Sep 10 18:10:10.335112 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:10.335010    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-twhhg_9c96a912-111d-4577-8119-73003a210757/main/0.log"
Sep 10 18:10:10.446669 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:10.446637    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-jlwql_7772eb5a-b6b8-499c-b70f-5856d1a8cb8f/main/0.log"
Sep 10 18:10:16.221907 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:16.221869    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:10:16.288119 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:16.288092    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log" limit=1024
Sep 10 18:10:19.137077 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:19.137031    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-79kh2_b00454b7-c84c-4280-87ca-39149012189f/global-pull-secret-syncer/0.log"
Sep 10 18:10:19.227989 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:19.227955    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4szqx_e5baaa61-1d31-4de2-b8bb-42cffda54965/konnectivity-agent/0.log"
Sep 10 18:10:19.342272 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:19.342245    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-165.ec2.internal_6fbc923c8aeb003a0cb6b765d09dd22b/haproxy/0.log"
Sep 10 18:10:24.210167 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:24.210134    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-856bfbb5f6-r8jtd_147919b5-2162-4add-9224-eb9f5f0ddf3c/authorino/0.log"
Sep 10 18:10:24.293058 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:24.293036    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-4z92c_8e2e0e76-065a-46cd-b9e5-768fa5263b10/kuadrant-console-plugin/0.log"
Sep 10 18:10:24.355376 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:24.355352    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-7g2kt_f608ec2a-3953-46e1-a116-79b27d453365/limitador/0.log"
Sep 10 18:10:24.435549 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:24.435521    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-nvcx5_cfce37ba-84e0-45b2-a10b-c7512c1cf106/manager/0.log"
Sep 10 18:10:26.078813 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.078780    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c9d989b66-nmpgs_ce5a5c11-d93e-4ca5-8f16-0393e2246481/metrics-server/0.log"
Sep 10 18:10:26.106736 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.106714    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-jjjcs_0fe4d4d9-cbea-4857-a3f0-5faacd4c0564/monitoring-plugin/0.log"
Sep 10 18:10:26.143775 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.143750    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9l5j_04051485-b57b-4e87-8e76-4ba3cd3d5721/node-exporter/0.log"
Sep 10 18:10:26.165945 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.165923    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9l5j_04051485-b57b-4e87-8e76-4ba3cd3d5721/kube-rbac-proxy/0.log"
Sep 10 18:10:26.187051 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.187035    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d9l5j_04051485-b57b-4e87-8e76-4ba3cd3d5721/init-textfile/0.log"
Sep 10 18:10:26.420206 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.420171    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cf7rr_3e5825a7-1574-44b5-8476-093ab3ca6d4c/kube-rbac-proxy-main/0.log"
Sep 10 18:10:26.448616 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.448596    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cf7rr_3e5825a7-1574-44b5-8476-093ab3ca6d4c/kube-rbac-proxy-self/0.log"
Sep 10 18:10:26.471612 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.471585    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-cf7rr_3e5825a7-1574-44b5-8476-093ab3ca6d4c/openshift-state-metrics/0.log"
Sep 10 18:10:26.519127 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.519058    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/prometheus/0.log"
Sep 10 18:10:26.544774 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.544744    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/config-reloader/0.log"
Sep 10 18:10:26.573732 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.573712    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/thanos-sidecar/0.log"
Sep 10 18:10:26.591212 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.591194    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/kube-rbac-proxy-web/0.log"
Sep 10 18:10:26.620347 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.620301    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/kube-rbac-proxy/0.log"
Sep 10 18:10:26.650608 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.650580    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/kube-rbac-proxy-thanos/0.log"
Sep 10 18:10:26.675973 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:26.675922    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f6dff5b7-7056-4a0d-9cce-ed0d6360b193/init-config-reloader/0.log"
Sep 10 18:10:29.301265 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.301219    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"]
Sep 10 18:10:29.301793 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.301615    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dde438dc-0636-4103-8a68-27643d5fa5df" containerName="maas-api"
Sep 10 18:10:29.301793 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.301630    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde438dc-0636-4103-8a68-27643d5fa5df" containerName="maas-api"
Sep 10 18:10:29.301793 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.301702    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="dde438dc-0636-4103-8a68-27643d5fa5df" containerName="maas-api"
Sep 10 18:10:29.304789 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.304765    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.307325 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.307286    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqhnx\"/\"openshift-service-ca.crt\""
Sep 10 18:10:29.307492 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.307326    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqhnx\"/\"kube-root-ca.crt\""
Sep 10 18:10:29.307492 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.307372    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqhnx\"/\"default-dockercfg-6n2nd\""
Sep 10 18:10:29.312412 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.312383    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"]
Sep 10 18:10:29.423117 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.423088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-sys\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.423117 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.423116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-podres\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.423320 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.423205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-lib-modules\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.423320 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.423257    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d54cm\" (UniqueName: \"kubernetes.io/projected/d275c50c-5b6a-4937-9779-85f03f3a5041-kube-api-access-d54cm\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.423320 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.423294    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-proc\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.523953 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.523920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-sys\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524087 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.523956    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-podres\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524087 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-lib-modules\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524087 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524044    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-sys\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524087 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524056    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d54cm\" (UniqueName: \"kubernetes.io/projected/d275c50c-5b6a-4937-9779-85f03f3a5041-kube-api-access-d54cm\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524087 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524087    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-proc\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524335 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524157    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-lib-modules\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524335 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-podres\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.524335 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.524199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d275c50c-5b6a-4937-9779-85f03f3a5041-proc\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.531504 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.531471    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d54cm\" (UniqueName: \"kubernetes.io/projected/d275c50c-5b6a-4937-9779-85f03f3a5041-kube-api-access-d54cm\") pod \"perf-node-gather-daemonset-r9f85\" (UID: \"d275c50c-5b6a-4937-9779-85f03f3a5041\") " pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.615936 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.615918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:29.680179 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.680156    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c69794f4-4m8cg_b8bf0a3a-6e81-490e-aad8-e6ce79315f81/console/0.log"
Sep 10 18:10:29.883210 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:29.883188    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"]
Sep 10 18:10:29.885592 ip-10-0-140-165 kubenswrapper[2571]: W0910 18:10:29.885566    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd275c50c_5b6a_4937_9779_85f03f3a5041.slice/crio-a9e4560c1996907f3e080b0cb95d1564e496931f6cc6ca98dd322ab4c2266460 WatchSource:0}: Error finding container a9e4560c1996907f3e080b0cb95d1564e496931f6cc6ca98dd322ab4c2266460: Status 404 returned error can't find the container with id a9e4560c1996907f3e080b0cb95d1564e496931f6cc6ca98dd322ab4c2266460
Sep 10 18:10:30.491376 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:30.491344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85" event={"ID":"d275c50c-5b6a-4937-9779-85f03f3a5041","Type":"ContainerStarted","Data":"620d506b77c69fd1f559e87a73293de3b742c9780582e52745e5f1861bd8feda"}
Sep 10 18:10:30.491824 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:30.491425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85" event={"ID":"d275c50c-5b6a-4937-9779-85f03f3a5041","Type":"ContainerStarted","Data":"a9e4560c1996907f3e080b0cb95d1564e496931f6cc6ca98dd322ab4c2266460"}
Sep 10 18:10:30.491824 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:30.491472    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:30.510002 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:30.509957    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85" podStartSLOduration=1.509942879 podStartE2EDuration="1.509942879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 18:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 18:10:30.507914038 +0000 UTC m=+2046.025594192" watchObservedRunningTime="2026-09-10 18:10:30.509942879 +0000 UTC m=+2046.027623103"
Sep 10 18:10:31.584383 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:31.584353    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fwlg9_49be2ede-2ac7-470e-86ea-613d99ff8052/dns/0.log"
Sep 10 18:10:31.604279 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:31.604256    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fwlg9_49be2ede-2ac7-470e-86ea-613d99ff8052/kube-rbac-proxy/0.log"
Sep 10 18:10:31.651591 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:31.651572    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pttp4_26515fca-4074-4d40-b42c-235ffebf01c3/dns-node-resolver/0.log"
Sep 10 18:10:32.227309 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:32.227284    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-crqvl_a7e5026e-a37f-45a4-a898-89f2c1052576/node-ca/0.log"
Sep 10 18:10:33.624466 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:33.624415    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-bstbn_807feed5-8bc2-4b14-82ab-0bfa85face7b/kube-auth-proxy/0.log"
Sep 10 18:10:34.693310 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:34.693279    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jzwqz_577be349-f6d9-41a3-b751-b53e6ccf096e/serve-healthcheck-canary/0.log"
Sep 10 18:10:35.200118 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:35.200093    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7hsh_f8f75fc7-1270-43c4-9efc-b9fc68497238/kube-rbac-proxy/0.log"
Sep 10 18:10:35.218731 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:35.218708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7hsh_f8f75fc7-1270-43c4-9efc-b9fc68497238/exporter/0.log"
Sep 10 18:10:35.238346 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:35.238317    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7hsh_f8f75fc7-1270-43c4-9efc-b9fc68497238/extractor/0.log"
Sep 10 18:10:36.507509 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:36.507484    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqhnx/perf-node-gather-daemonset-r9f85"
Sep 10 18:10:38.079959 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:38.079924    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-m897n_c6b2dca6-2edb-4cc1-a39a-22426bf01dd4/manager/0.log"
Sep 10 18:10:38.097922 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:38.097901    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbf885bd4-m897n_c6b2dca6-2edb-4cc1-a39a-22426bf01dd4/copy-manifests/0.log"
Sep 10 18:10:39.665255 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:39.665213    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57bb875478-5s8mk_860ddcbd-d13e-439a-bd1f-e1392d1fd340/manager/0.log"
Sep 10 18:10:40.240654 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:40.240622    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-bstbn_807feed5-8bc2-4b14-82ab-0bfa85face7b/kube-auth-proxy/0.log"
Sep 10 18:10:42.316077 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:42.316050    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cd5576685-bstbn_807feed5-8bc2-4b14-82ab-0bfa85face7b/kube-auth-proxy/0.log"
Sep 10 18:10:46.249019 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.248946    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/kube-multus-additional-cni-plugins/0.log"
Sep 10 18:10:46.268728 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.268684    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/egress-router-binary-copy/0.log"
Sep 10 18:10:46.290264 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.290241    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/cni-plugins/0.log"
Sep 10 18:10:46.309859 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.309836    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/bond-cni-plugin/0.log"
Sep 10 18:10:46.330472 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.330424    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/routeoverride-cni/0.log"
Sep 10 18:10:46.348931 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.348910    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/whereabouts-cni-bincopy/0.log"
Sep 10 18:10:46.367413 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.367395    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t8l7c_b135c9cb-01d1-4ac5-a31f-720cc697b74b/whereabouts-cni/0.log"
Sep 10 18:10:46.418876 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.418847    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rtggj_6adb3bcb-c09a-4291-8509-84188419571e/kube-multus/0.log"
Sep 10 18:10:46.588869 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.588794    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-czh2q_22631be7-f152-457e-a647-4dda83f32fa9/network-metrics-daemon/0.log"
Sep 10 18:10:46.604266 ip-10-0-140-165 kubenswrapper[2571]: I0910 18:10:46.604236    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-czh2q_22631be7-f152-457e-a647-4dda83f32fa9/kube-rbac-proxy/0.log"